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ABSTRACT 


The  restoration  of  color  errors  in  diqitally  recorded 
color  images  is  considered  in  this  dissertation.  A vector 
space  model  of  a qeneral  digital  color  imaqe  recordinq 
system  is  derived  and  the  eauations  representing  the  model 
and  the  eauations  of  colorimetry  are  expressed  in  matrix 
form.  Computer  algorithms  are  derived  which  correct  color 
errors  introduced  by  imoer tections  in  the  color  recorJing 
system.  The  sources  of  color  error  which  are  considered 
include  sensor  spectral  responses  which  depart  from  ideal 
color  matching  curves,  crosstalk  between  color  signal 
channels,  aid  system  nonlinearities.  The  special  case  of  a 
color  film-diqital  scanner  system  is  examined  in  detail, 
although  the  methods  derived  apply  to  a wider  class  of 
color  or  multispectral  sensing  and  recording  systems.  The 
success  of  the  correction  algorithms  is  demonstrated  using 
a computer  simulation  of  the  film-scanner  system.  The 
algorithms  for  correction  of  spectrally  imperfect  sensors 
are  also  tested  using  a specially  created  six  band 
multispectral  test  imaqe. 

The  generalized  matrix  inverse  is  used  extensively  in 
this  report.  Least  sauares,  minimum  norm,  and  Wiener 
estimation  algorithms,  in  the  form  of  generalized  inverses, 
are  applied  to  the  correction  of  sensor  imperfections.  The 
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Utility  of  the-  generalized  inverse  in  the  spectral  domain 

is  also  demonstrated  bv  applving  it  to  some  related  color 

problems.  These  .nclude  the  estimation  of  the  spectral 

response  of  a sensor  from  sample  readinqs,  and  computer 

generation  of  spectral  waveforms  with  desired  colorimetric 
properties . 
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1.  Introduction 

This  report  concerns  the  application  of  sampled  data 
processing  and  estimation  methods  to  the  restoration  of 
digitally  recorded  color  images.  The  restoration  of  color 
values  which  have  been  degraded  by  sensor  and  storage 
imperfections  is  an  objective  which  has  been  pursued  since 
color  image  recording  systems  were  invented.  The  methods 
of  color  restoration  which  have  been  heretofore  explored 
have  varied  with  each  method  of  color  image  recording.  The 
first,  and  by  far  the  most  common,  medium  of  color  image 
recording  is  color  photographic  film.  Most  color  film 
images  do  not  undergo  any  post-development  restoration  at 
all.  Those  film  images  which  are  restored  are  usually 
modified  only  by  photographic  masking,  which  attempts  to 
provide  pleasingly  saturated  colors  and  a proper  white 
balance.  Masking  and  other  photographic  restoration 
methods  are  extremely  limited  in  their  ability  to  correct 
color  errors.  They  are  slow,  inflexible  and  usually  apply 
the  same  correction  to  all  points  in  the  image.  When  color 
values  are  converted  to  electrical  signals,  as  in  color 
television  and  in  color  scanning  devices,  much  more  can  be 
done  to  correct  color  errors,  often  in  a real-time 
interactive  mode.  Here  again,  the  object  is  most  commonly 
the  production  of  a pleasing  saturated  picture,  not  the 
increase  of  color  fidelity  as  measured  by  a Quantitative 
criterion. 
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With  the  comparatively  recent  development  of  digital 
image  recording,  it  becomes  possible  to  extend  further  the 
capabilities  of  color  image  restoration.  All  the  power  and 
flexibility  of  the  stored  program  digital  computer,  which 
have  been  proved  as  successful  in  the  restoration  and 
enhancement  of  digital  monochrome  imagery,  can  be  applied 
to  color  image  restoration.  The  digital  computer  can  be 
utilized  in  the  optimization  and  testing  of  restoration 
algorithms  as  well  as  in  the  execution  of  these  algorithms 
to  restore  large  quantities  of  digital  color  imagery. 


This  report  will  examine  the  computer  application  of 
some  standard  mathematical  tools  and  operations  to  the  task 
of  restoring  errors  introduced  by  sensor  and  storage 
imperfections  in  a general  color  image  recording  system. 
The  system  imperfections  which  are  considered  include 
non-ideal  sensor  spectral  responses,  crosstalk  between 
color  signal  channels,  and  errors  introduced  by  system 
nonlinear ities.  A color  film-electronic  scanner  color 
system  is  analyzed  in  some  detail,  but  the  methods 
developed  can  be  applied  to  a wider  class  of  color 
analyzing  and  recording  systems.  In  addition,  some  of  the 
mathematical  restoration  tools  utilizing  generalized 
inverses  are  applied  to  other  color  problems  such  as  the 
spectral  calibration  of  color  scanners  and  the  computer 
generation  of  spectral  waveforms  with  desired  colorimetric 
properties. 
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The  dissertation  consists  of  seven  chapters.  Chapter 
2 presents  a short  tutorial  on  colorimetry,  color 
reproduction  and  restoration.  Chapter  3 discusses  the 
advantages  of  digital  color  restoration  and  vector  space 
modelling  of  color  systems.  Tristimulus  estimation  using 
linear  estimation  methods  is  treated  in  Chapter  4.  Chapter 
5 discusses  methods  of  inverting  the  nonlinear  film-scanner 
equations  in  order  to  estimate  film  exposure  values.  Some 
related  applications  of  digital  color  methods  are  treated 
in  Chapter  6 and  some  conclusions  and  general  remarks  about 
the  dissertation  are  presented  in  Chapter  7. 


2.  Color  Reproducti* 


The  long  established  field  of  color  science 
historically  has  consisted  of  two  branches:  colorimetry, 
whici:  is  the  measurement  of  color,  and  color  reproduction. 
These  two  areas  of  study  have  been  related  only  slightly 
ftnd,  for  two  reasons,  have  not  been  put  on  a rigorous  basis 
until  recent  years.  First,  established  color  standards 
came  into  use  only  after  the  formation  of  the  Commission 
Internationale  d'Eclairage  (CIE)  in  the  early  1930's. 
Secondly,  until  the  development  cf  color  television  in  the 
1950  s,  color  reproduction  was  achieved  by  photographic 
methods.  Because  of  the  complexity  of  subtractive  color 
photography,  its  capabilities  were  advanced  primarily  by 
empirical  means,  and  not  through  the  use  of  colorimetry, 
which  is  based  on  the  simpler  theory  of  additive  primary 
colors.  in  the  last  two  decades,  color  reproduction 
technology  has  adopted  the  terminology  and  methods  of 

colorimetry,  and  the  two  branches  of  color  science  have 
be ;ome  more  unified. 

In  Chapter  2 which  follows,  three  topics  will  be 
discussed.  First,  the  basics  of  colorimetry  win  be  oiven, 
including  a brief  discussion  of  color  error  formulas. 
Second,  the  principles  of  color  reproduction  and  the 
important  sources  of  reproduction  errors  will  be  described. 
Finally,  some  of  the  standard  techniques  of  color 
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restoration,  both  photographic  and  electronic,  will  be 
discussed . 

2.1  Colorimetry 

Colorimetry,  the  science  of  color  measurement,  is 
based  on  the  experimental  observation  that,  over  some  range 
of  colors  and  observing  conditions,  any  « olor  stimulus 
(defined  as  radiant  flux  detected  by  an  observer's  visual 
system  and  evoking  a sensation  of  color  [1])  can  be  matched 
by  an  additive  mixture  of  three  fixed  primary  coloi 
stimuli . 

The  experimental  laws  governing  the  relations  between 
the  spectral  intensity  distribution  of  a color  stimulus  and 
the  amounts  of  each  primary  color  stimulus  required  to 
match  it  form  the  basis  of  colorimetry.  The  fundamental 
color  matching  equations  are 

few  SR(X)  dx  = /cp(X)  SR(X)  dX 

/c«)  SG(X)  dx  = J Cp(X)  SG(X)  dx  (2-1) 

/c«>  SB(X)  dX  = J" Cp (X)  Sp(X)  dX 


where  C(x)  is  the  spectral  intensity  distribution  of  the 
color  to  be  matched  and  Cp  ( X)  is  the  spectral  intensity 
distribution  of  the  weighted  sum  of  primary  colors  which 


matches  C(X).  The  characteristics  of  the  standard  average 
observer  with  normal  color  vision  are  given  by  the  spectral 
sensitivity  curves  SR(X),  SG(X),  and  S (x  ) and  the  spectral 
intensity  of  the  matching  primaries  are  P ( X ) , p ( X)  and 
Pg(M  . The  primary  colors  are  usually  suitably  chosen  red, 
green,  and  blue  lights. 

If  the  weights  or  amounts  of  each  primary  which 
together  match  C ( x)  are  given  by  AR(C)  , AG(C)  and  AB(C)  , 
then 

CP(X)  = VC)  PR(X)  + ag(c)  pg(X)  + Ap(c)  pb(x)  (2-2) 

Then  if  IR,  Iq,  iR  are  defined  as  below,  a color  match 
requires 

IR4/c(X)SR(X)dX=y[AR(C)PR(X)+AG(C)PG(X)+AB(C)PB(X)]SR(X)dX  (2-3a) 

lG^JC(''  )SG(X)JX^AR(C)PR(X)+AG(C)PG(X)+ABrC)PB(X)^SG(X)dX  ( 2~3b) 

IB4/c(X)SB(X)dX*yiAR(C)PR(X)+AG(C)PG(X)+AB(C)PB(X)]SB(X)dX  (2-3c) 

I 

I 

whei.o  A (C)  , A (C)  , A (C)  are  called  the  primary  matching 
values  of  the  color  C.  When  these  primary  matching  values 
are  normalized  by  the  primary  matching  values  of  a 
reference  white  color  W,  the  resulting  normalized  values 
are  called  the  tristimulus  values  of  C 
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t2(c) 


t3(c) 


v w 

( 2-4a) 

AX(W) 

A2(C) 

( 2-4b) 

a2(w) 

A-(C) 

A,(W) 

( 2-4c) 

Equations  (2-3)  and  (2-4)  can  be  combined  in  matrix  form  to 
give  the  following  equation  of  color  matching 
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where 


(2-5) 


= Pj(X)  St(X)  dX 

and  in  the  subscripts  i,j  R=l,  G=2,  B=3. 


(2-6) 


The  color  matching  equations,  expressed  above  in 
different  but  equivalent  eauations  (2-1),  (2-3),  and  (2-5), 
illustrate  the  main  objective  of  colorimetry:  to  determine 
whether  or  not  two  given  stimuli,  C(A)  and  Cp(A),  match  in 
color.  The  two  colors  match,  color imetrically,  when  their 
tristimulus  values  are  equal. 

It  is  important  to  observe  that  colorimetry  does  not 
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attempt  to  characterize  the  sensation  which  an  observer 
perceives  when  he  receives  a color  stimulus.  To  do  this 
would  require  a much  more  complex  model  involving  spatial 
and  temporal  parameters  relating  to  state  adaption,  stimuli 
surrounding  the  test  stimuli  and  other  parameters. 
Colorimetry  is  not  designed  to  provide  a general  color 
perception  model,  only  to  answer  the  simpler  but  still 
important  question  of  whether  two  colors  match  or  not. 

The  presence  of  a linear  integration  of  spectral 
intensity  over  wavelength  in  the  color  matching  equations 
is  a reflection  of  the  underlying  assumptions  of  linearity 
and  additivity  in  color  matching.  Linearity  and 
additivity,  which  hold  accurately  over  a wide  range  of 
observing  conditions  [1],  require: 

(1)  that  the  match  between  any  two  colors  continues  to 
hold  when  the  spectral  distributions  of  the  two  colors  are 
each  scaled  up  or  down  by  the  same  constant  factor;  and, 


*2)  that  if  colors  A and  B match,  and  if  colors  C and 

D match  then  the  additive  color  mixtures  (A+C)  and  (B+D) 
also  match. 

Under  these  assumptions,  it  is  easily  seen  that  any 
color  C ( A ) may  be  thought  of  as  a weighted  sum  of  spectral, 
i.e.,  monochromatic,  colors.  Because  of  linearity  and 
additivity,  the  tristimulus  values  of  the  spectral 

8 


II. 


distribution  C(A)  can  be  obtained  by  integrating  the 
tristimulus  values  o.'  the  spectral  components  of  C(A)  over 
all  wavelengths.  Combining  eqs.  (2-1),  (2-3)  and  (2-5) 

gives 


Tj (C)  = C (X)  TX(X)  dX 
T2(C)  = C(X)  T2(X)  dX 


T3(C)  = C(X)  T3(X)  dX 


where 
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sGU) 
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(2-7) 


(2-8) 


The  functions  Tj (A)  , T2(A),  T3(A)  are  called  the  color 

matching  curves  of  the  set  of  primaries  P (A)  , p (A)  , 

R G 

pb(M  • These  curves  completely  describe  human  visual  color 
response  under  the  colorimetric  model  by  determining 
through  eq.(2-7)  the  tristimulus  values,  or  primary 
weights,  needed  to  match  any  color.  It  will  subsequently 
be  shown  that  Tj(A),  T2(X),  T2(X)  are  the  optimum  spectral 
sensitivities  for  a color  sensor  whose  outputs  directly 
drive  display  primaries  P (A) , P_(\)  , P_ ( A ) . Figure  (2-1) 

R Ci  B 

contains  a plot  of  the  color  matching  curves  corresponding 
to  equal  radiance  narrowband  primaries  at  700.0,  546.1,  and 


i 

J 
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435.8  nanometers,  with  a reference  white  whose  power 
spectral  density  is  constant  with  wavelength  (equal  energy 
white)  . 

The  color  matching  curves  of  eq.  (2-8)  depend  on  the 
matching  primaries  P (A),  P_(A),  P„(A)  only  through  the 
matrix  D= ( d_  ] of  eq.  (2-6).  Therefore,  the  three  color 
matching  curves  corresponding  to  a particular  primary  set 
may  be  derived  from  those  corresponding  to  another  primary 
set  by  a 3x3  matrix  multiplication,  so  that  each  new  curve 
is  a weighted  sum  of  the  three  original  color  matching 
curves.  It  follows  from  eq.  (2-7)  that  the  tristimulus 
values  of  a color  with  respect  to  the  new  primaries  are 
equal  to  the  old  tristimulus  values  multiplied  by  the  same 
3x3  matrix. 


It  is  of  interest  to  note  that  color  matching  curves 

T 1 ( X ) , T^(A) , T^(A)  are  obtained  directly  by  means  of  color 

matching  experiments  with  test  subjects,  rather  than  by 

calculating  them  from  the  visual  sensitivity  curves  S , S , 

R G 

Sgthrough  eq.  (2-8).  The  visual  sensitivity  curves  are, 
even  today,  known  with  less  accuracy  than  are  the  color 


matching  curves, 

since  it  is  difficult 

to  measure 

them 

directly.  The 

responses  SR,  SG,  and  SB  cannot 

be 

determined  from 

the  color  matching  curves 

via  eq. 

(2- 

-8) 

since  SR,  SG, 

SR  are  contained  in  each 

element  d. . 

of 

D. 

For  this  reason, 

color  matching  is  nearly 

always  done 

by 
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comparing  the  tristimuluL  values  of  two  colors  obtained 
from  equations  of  the  form  of  eq.  (2-7)  rather  than  by 

using  the  spectral  sensitivities  of  the  eye  directly,  as  in 
eqs . (2-1). 

ne  determination  of  ti/istimulus  values  of  eas.  (2-7) 
requires  that  Tj  (A)  , (A)  , t3  (A)  represent  a par ticular 

set  of  "standard  observer"  color  matching  curves  such  as 
the  Cl E 1931  standard  observer  curves  X(A),  Y(\),  Z(\) 

shown  in  figure  (2-2).  in  this  case,  the  tristimulus 

equations  become 

X = rJc(\)  x(X)  dX 

Y = r/c(X)  Y(X)  dX  (2-9) 

z = r/c(X)  Z (X ) dX 

where  R may  be  chosen  arbitrarily  when  only  the  relative 
tr istimulus  values  are  of  interest,  when  R is  chosen  to  be 
R=K  ~680  lumens/watt,  the  value  of  Y is  then  the  luminous 
flux,  in  lumens,  of  the  color  C(A). 

When  a color  consists  of  radiant  flux  reflected  (or 
transmitted)  by  a nonradiating  object,  it  is  called  an 
object-color  stimulus.  a reflected  object  color  is 
specified  by  the  spectral  distribution  P(A)  i(\)  where  P(\) 
is  the  spectral  reflectivity  and  I (X)  is  the  spectral 
intensity  (power)  distribution  of  the  light  source.  The 
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tristimulus  equations  are  then 

X = rJ?(\)  1(A)  x(A)  dA 

Y = rJ?(X)  1(A)  Y(A)  dA 
z = r/p ( A ) 1(A)  Z (A)  dA 


(2-10) 


where  R is  usually  chosen  so  that  for  a perfectly 
reflecting  and  perfectly  diffusing  object  (P(A)=1.0  at  all 
wavelengths) , the  luminance  Y has  a value  of  100.  This  may 
be  ensured  by  defining  R as 


fl(A)  Y(A ) dA 


(2-11) 


The  spectral  distribution  of  the  illuminant,  1(A),  is 
usually  assumed  to  be  one  of  the  CIE  standard  illuminants: 
CIE  illuminant  A represents  a black-body  emitter  at 
2856  K (approximates  the  spectral  distribution  of  an 
incandescent  light  bulb) . 

CIE  illuminant  C represents  average  daylight  with  a 
correlated  color  temperature  of  approximately  6770  K. 
CIE  illuminant  D65  represents  a phase  of  daylight  with 
a correlated  color  temperature  of  approximately  6500  K 
(using  more  recent  measurements  than  those  of 

illuminant  C) . 

CIE  illuminant  D55  and  D75  represent  phases  of 

daylight  with  correlated  color  temperatures  of  5500 
and  7500  K,  respectively. 

The  spectral  distributions  of  these  illuminants  are  shown 
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in  figure  U-3)  . 


Although  any  color  stimulus  is  uniquely  specified 
color imetr ically  by  giving  its  tristimulus  values  in  any  of 
the  standard  observer  coordinate  systems,  it  is  more 
convenient  for  plotting  purposes  to  reduce  the 
dimensionality  of  a color  from  three  to  two  by  normalizing 
each  tristimulus  value  by  the  sum  of  the  three  tristimulus 
values.  These  normalized  values  are  called  chromaticity 
values  and  are  represented  by  the  lower  case  symbol  of  the 
corresponding  tristimulus  symbol.  For  example, 


T,  + T„  + T„ 


(2-12) 


for  i = l , 2 , 3 or 


x = 


y = 


x + y + z 

Y 


z = 


X + Y + Z 
Z 


(2-13) 


X + Y + Z 


Sine;:  x+y+z=l,  only  two  chromatici  ties , usually  x and  y, 
need  be  specified.  When  a third  number  is  required  to 
complete  the  specification  of  the  color,  the  luminance 
value  Y is  often  given  also.  The  complete  color 
specification  is  then  x,y,Y. 


Two  chromaticity  values  may  be  used  as  rectangular 
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coordinates  in  a chromaticity  diagram.  The  CIE  1931  (x,y) 
chromaticity  diagram  is  shown  in  figure  (2-4)  with  the 
locus  of  spectral  (monochromatic)  colors  from  X =380 
nanometers  to  X =780  nanometers  included.  It  may  be  shown 
easily  that  the  location  on  a chrumaticity  diagram  of  a 
positively  weighted  sum  of  two  color  stimuli  must  lie  on 
the  straight  line  connecting  the  chromaticity  coordinates 
of  the  two  color  stimuli.  It  follows  that  any  realizable 
color,  i.e.,  a positively  weighted  sum  of  spectral  colors, 
must  lie  within  the  locus  of  spectral  colors  on  a 
chromaticity  diagram.  When  tristimulus  values  are 
normalized  so  that  (1,1,1)  corresponds  to  a reference 
white,  the  reference  white  is  then  located  at  (1/3, 1/3)  on 
the  chromaticity  diagram. 

Color  Difference  Formulas 

It  was  previously  stated  that  colorimetry  attempts  to 

answer  the  question  of  whecher  two  colors  appear  to  a human 

observer  to  match  or  not.  Two  colors  match 

colonmetricallv  when  their  tristimulus  values  are  equal. 

The  definition  of  colorimetry  can  be  broadened  to  include 

the  measurements  of  the  color  difference  when  the 

tr istimulus  values  of  two  colors  are  not  equal.  A formula 

is  desired  which  operates  on  the  tristimulus  values  of  two 

colors  and  produces  a number  which  is  proportional  to  the 

perceived  difference  between  the  colors  for  an  average 
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observer  with  normal  color  vision.  Obtaining  such  a 
formula  is  difficult  since  the  subjective  estimation  cf 
color  differences  by  an  observer  is  far  less  accurate  than 
is  strict  color  matching,  and  the  influence  of  viewing 
conditions  is  very  strong  [2]. 

Still,  there  are  several  useful  color  difference 
formulas  based  on  the  1931  CIE  standard  observer  (XYZ) 
system.  One  system  which  attempts  to  provice  a 
chromaticity  diagram  in  which  equal  distances  correspond 
approximately  to  equal  visual  differences  is  the  1960  CIE 
uniform  chromaticity  scale  (UCS)  diagram.  This  system  is 
derived  from  a projective  transformation  of  the  1931  cie 
chromaticity  diagram  [3].  The  transformation  equations  are 

u M 

X + 15Y  + 3Z 

(2-14) 

6Y 

V X + 15Y  + 3Z 

where  u,v  are  UCS  chromatici ties  and  the  UCS  tristimulus 
values  are  called  U,V  and  W.  The  UCS  c aromaticity  diagram 
showing  the  locus  of  spectral  colors  is  given  in  figure 
(2-5).  The  1960  CIE-UCS  system  has  been  extended  to 
include  luminance  differences  in  the  U*,V*,W*  system  [4], 
defined  by 
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W*  = 25Y1/3  - 17 

u*  = 13W(u  - u ) 
o 

tn 

rH 

1 

rsi 

V*  = 13W(v  - V ) 
o 

for  1<Y<100, where  u0,vo  are  the  UCS  chromaticities 
reference  white.  The  resulting  color  difference 

for  colors  U *,V  *,W  * and  U *,V  *,w  * is  given  by 

of  the 
formula 

AE  = [ (U*  - U*)2  + (V*  - V*)2  + (W*  - W*)2]1/2 

(2-H, 

Another  widely  used  uniform  color  coordinate  system  is 

'"he  cube  r°ot  system  [5].  The  formulas  for  the  cube  root 
coordinates  L,  a,  b are  given  by 

L « 25.29G1/3  - 18.38 
a = 106.0  (R1/3  - G1/3) 
b = 42.34  (G1/3  - B1/3) 

where  R-1.02  X,  G=Y,  and  8=0.847  Z.  The  color  distance 


formula  is  then  the  Euclid 


ean  distance  in  L,  a,  b space 


AE  = [ (AL) 2 + (Aa)2  + (Ab) 2]1/2 


(2-17) 


In  each  of  the  two  systems  described  above,  the  U*V*W* 

and  the  L a b systems,  one  coordinate  (W*  or  L)  corresponds 

to  the  perceived  brightness  of  the  color,  while  the  other 

two  coordinates  form  a plane  in  which  the  origin 

corresponds  to  an  achromatic  grey.  m each  system,  color 
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saturation  increases  with  distance  from  the  origin,  while 
hue  or  dominant  wavelength  varies  with  angle  about  the 
origin.  In  all  such  perceptually  uniform  systems,  the 
coordinates  are  nonlinear  functions  of  the  tristimulus 
values,  while  the  tristimulus  values  are  always  linear 
functions  of  the  color  spectral  distribution  C(\). 


2.2  Color  Reproduction 


In  this  section,  additive  and  subtractive  color 
reproduction  will  be  described  using  the  methods  and 
expressions  of  colorimetry.  Colorimetry  possesses  serious 
limitations  as  a basis  for  the  reproduction  of  color  images 
which  are  large,  spatially  complex,  and  possibly  time 
varying.  Color  matching  curves,  the  basic  functions  of 
colorimetry,  are  obtained  by  the  matching  of  two  small, 
uniform  patches  of  color  which  are  of  three  to  ten  degrees 
in  angular  size,  seen  against  a black  surround.  It  is  well 
known  that  perceived  color  sensations,  and,  to  a lesser 
extent,  color  matches,  change  when  the  surrounding  area  of 
the  test  colors  is  changed  visually  [6].  Still, 
colorimetry  has  served  as  the  foundation  of  color 
television  design,  in  the  absence  of  a better  visual  model, 
and  the  results  have  been  quite  satisfactory  [7] . 


Color  reproduction  methods  which  are  based  on  the 
rules  of  colorimetry  attempt  to  satisfy  the  following  two 
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conditions: 

(1)  The  spectral  sensitivities  of  the  color  sensors 
are  a set  of  color  matching  curves. 

(2)  The  sensor  outputs  drive  display  primaries  which 
are  the  primaries  corresponding  to  the  sensor  color 
matching  curves  (or  the  sensor  outputs  have  been 
transformed  to  correspond  to  the  display  primaries). 

The  two  most  important  sources  of  color  reproduction 
errors  occur  when  the  two  above  conditions  are  not  met. 
Imperfect  spectral  sensitivities  and  deficiencies  of  the 
primaries  are  present,  to  some  extent,  in  all  color 

reproduction  systems,  whether  photographic,  television,  or 
lithographic. 

Iiroerfect  spectral  sensitivities  occur  for  two 
reasons.  First,  real  sensors  such  as  photographic 
emulsions  and  vidicon  tubes  do  not  have  sensitivities  which 
are  color  matching  curves,  though  appropriate  optical 
filtering  may  allow  an  arbitrarily  close  fit  to  color 
matching  curves,  at  the  cost  of  some  loss  of  system 
sensitivity  and  signal-to-noise  ratio.  A second  reason  is 
that  spectral  sensitivities  may  be  intentionally  degraded 
in  order  to  compensate  partially  for  errors  introduced  in 
other  parts  of  the  system.  An  example  of  this  is  the  wide 
spectral  spacing  of  the  peak  sensitivities  of  the  three 
layers  of  some  color  films.  The  widely  spaced 
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sensitivities  introduce  an  exaggerated  saturation  of  colors 
which  helps  to  compensate  for  loss  of  saturation  in  the 
developed  transparency  caused  by  undesired  layer 
absorptions.  Undesired  absorptions  will  be  treated  at 
greater  length  later. 


A quantitative  figure  of  merit  for  color  sensors  is 
the  colorimetric  quality  factor  q defined  by  Neugebauer 
[8] . The  quality  factor  of  a spectral  sensitivity  curve  is 
unity  when  the  curve  is  any  color  matching  curve.  The 
quality  factor  diminishes  as  the  sensitivity  curve  departs 
more  and  more  from  its  nearest  (in  the  least  squares  sense) 
color  matching  curve.  Tne  quality  factor  a is  obtained  by 
forming  a collection  of  functions  from  three  color  matching 
curves  and  any  complete  orthonormal  set,  such  as  the  usual 
set  of  sine  and  cosine  functions.  This  collection  is 
or thonor mal i zed  by  the  Gram— Schmitt  process  giving  an 
orthonormal  function  set 


VX>«  U2(X)’  U3(X)» 


, Un(A), 


the  first  three  of  which  are  color  matching  curves.  Such 
an  orthonormal  set  is  shown  in  figure  (2-6).  Figure  (2-6a) 
shows  the  first  three  functions,  all  color  matching  curves. 
The  first  (and  only  nonnegative)  member  of  the  set  is  the 
luminous  efficiency  curve.  The  second  three  functions  of 
the  set,  derived  using  sine  and  cosine  functions,  are  shown 
in  figure  ( 2-6b) . 
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The  given  filter  or  sensor  function  f ( X ) is  expanded 
in  this  orthonormal  set,  so  that 


f - l w 

1=1 


(2-18) 


where 


ft  - <f  Ut>  i/f(X)  Ui(X)  dX 


(2-19) 


the  quality  factor  a is  then  defined  to  be  the  ratio  of  the 
energy  of  f in  the  first  three  coefficients  of  (2-18)  to 
the  total  energy  of  f as  given  by 


Q i 


3 

l V 

1=1  1 

00 

1 V 

1=1 


2 2 2 
fl  +f2  +f3 

<f2> 


(2-20) 


A useful  property  of  the  representation  of  eq.  (2-18)  is 
that  the  least-squares  nearest  color  matching  curve  to  f(X) 
may  be  obtained  by  summing  the  first  three  components  of 
the  expansion 


3 

l 


1=1 


fi  Ui 


(2-21) 


The  value  of  the  quality  factor  q required  for  a color 

sensor  depends  on  the  application  of  the  sensor.  The 
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sensors  in  colorimeters  usually  have  q factor  values  of 
0.99  or  higher,  achieved  by  careful  adjust  ent  of  filter 
characteristics.  in  color  television  cameras,  the  q 
factors  of  the  three  color  sensors  are  also  high,  perhaps 
0.90  or  better.  The  sensor  curves  are  usually  chosen  to  be 
close  to  the  color  matching  curves  corresponding  to  the 
standard  NTSC  display  primaries,  where  the  negative  lobes 
of  the  ideal  curves  are  neglected  [9].  m color 
photography,  the  film  layer  sensitivities  may  have  g factor 
values  as  small  as  0.6,  or  even  less,  because  colorimetric 
accuracy  may  have  been  sacrificed  in  favor  of  other 
properties,  as  mentioned  earlier. 

The  second  major  source  of  errors  in  color 
reproduction  is  the  problem  of  cross-talk  or  undesired 
absorptions.  In  color  television,  this  might  be  caused  by 
imperfect  signal  decoding,  so  that  the  signal  driving  the 
red  display  primary  might  be  contaminated  by  the  blue  and 
green  display  signals.  Misalignment  in  a shadow  mask 
picture  tube,  allowing  the  electron  beam  of  one  primary  gun 
to  illuminate  phosphors  of  the  other  two  primaries,  is 
another  example  of  crosst^k  in  color  television,  m color 
photography  and  color  lithography  the  equivalent  problem  is 
called  undesired  absorption.  A color  film  transparency  has 
a cyan  (red-absorbing)  layer  which  should  control  the  red 
transmission  of  the  transparency,  with  the  other  two  layers 

(magenta  and  yellow)  absorbing  no  red  light  at  all.  in 
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reality,  red  light  is  absorbed  to  some  extent  by  the 
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magenta  and  yellow  layers.  Figure  (2-7)  shows  the 
spectrophotometr ic  absorption  curves  of  a typical  set  of 
photographic  dyes.  The  cyan  and  magenta  layers  absorb 
strongly  in  the  blue  (400-460  nm)  region,  where  ideally 
their  optical,  densities  should  be  zero.  The  cyan  dye  also 
exhibits  unwanted  absorption  in  the  green  portion  of  the 
spectrum.  In  each  case,  one  of  the  color  signals,  red, 
green,  or  blue,  is  contaminated  by  one  or  both  of  the  other 
two  color  signals.  An  ideal  set  of  color  reproduction  dyes 
would  be  similar  to  those  illustrated  in  figure  (2-8) . The 
rectangular  spectral  passband  maximizes  energy 
transmittance  for  a given  dye  saturation  purity,  while  the 
non-overlapping  stopbands  remove  the  undesired  absorption. 
Each  dye  affects  one  and  only  one  spectral  region  and  the 
resulting  ideal  subtractive  color  system  is  equivalent  to 
an  additive  color  system.  Since  unity  spectral 
transmittance  or  reflectance  is  not  achieved  by  real  dyes 
at  any  wavelength,  urdesired  absorption  is  always  present 
in  any  Leal  subtractive  color  reproduction  system. 

The  result  of  crosstalk  or  undesired  absorption  is 
color  error,  usually  a loss  of  saturation  as  a stronq 
signal  in  one  color  channel  excites  the  other  two  channels 
to  some  extent,  tending  to  neutralize  the  saturation  of  the 
color  at  the  display. 
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Figure  (2-7)  Photographic  dye  spectral  absorpti 


In  addition  to  sensor  imperfections  and  crosstalk, 
there  are  several  other  sources  of  color  errors  which  are 
of  lesser  importance,  but  which  cannot  be  neglected  in  a 
general  discussion  of  color  errors.  One  such  source  of 
errors  is  the  limiting  of  color  signals  by  one  or  more 
components  in  the  color  system.  Examples  include  dynamic 
range  limitations  in  a color  television  camera  tube  or 
display  and  the  limited  range  of  film  densities  in  a color 
photographic  system.  The  effects  of  limiting  may  be  seen 
by  examining  the  density  versus  log  exposure  curve 
(sometimes  called  the  Hurter  and  Driffield  curve  or  H and  D 
curve)  of  a typical  reversal  color  film,  shown  in  figure 
(2-9) . When  the  exposure  values  of  the  three  layers  all 
lie  to  left  of  the  upper  bend  in  the  H and  D curve,  the 
densities  limit  at  "black"  and  all  color  information  is 
lost  (Case  A) . When  all  three  exposure  values  lie  to  the 
right  of  the  right  bend  in  the  D log  E curve,  the  densities 
limit  at  "white"  (Case  B) . When  at  least  one  exposure 
value  limits  while  one  or  two  exposure  values  lie  on  the 
linear  portion  of  the  D log  E curve,  the  output  color  will 
not  be  neutral  and  its  hue  and  saturation  will  in  general 
exhibit  large  errors  (Case  C) . Only  when  all  three 
exposure  values  at  an  image  point  lie  on  the  linear  portion 
of  their  respective  curves  is  it  possible  to  maintain 
accurate  brightness,  hue  and  saturation  values  relative  to 
other  properly  exposed  points  in  the  image  (Case  D) . 
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Another  source  of  color  error  is  gamut  limitations  in 
the  display.  It  was  previously  stated  that  the  position  on 
a chromaticity  diagram  of  a weighted  sum  of  two  primary 
lights  lies  on  the  line  connecting  the  position  of  the 
primaries.  It  follows  that  the  reproducible  gamut  of  a 
three  primary  display  must  lie  within  the  triangle  on  the 
chromaticity  diagram  defined  by  the  positions  of  the  three 
primaries.  In  additive  displays  such  as  television 
monitors,  the  color  gamut  is  great  enough  to  include  all 
but  a few  relatively  infrequent  highly  saturated  colors. 
Subtractive  photographic  systems  are  further  restricted  in 
gamut  by  the  undesired  dye  absorptions  with  a resulting 
inability  to  reproduce  saturated  colors  at  high  lightness 
levels  [10] . 


2.3  Restoration  of  Color  Errors 


Restoration  of  color  errors  caused  by  system 

imperfections  of  the  types  discussed  in  the  previous 
section  is  conventionally  done  in  one  of  two  ways, 

depending  on  whether  the  color  reproduction  system  is 
additive  or  subtractive.  Color  television  is  the  only 
widely  used  additive  color  imaging  system.  In  color 
television,  undesired  spectral  absorption  cannot  take 
place,  and  its  additive  equivalent,  crosstalk  among  the 
three  color  signals,  is  negligible  when  the  signal  decoding 
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and  shadow  mask  alignment  in  the  receiver  are  correct. 
Imperfect  spectral  sensitivities  of  the  camera  tube  are 
always  present,  however,  and  some  kind  of  signal  matrixing 
(11,12,13]  is  usually,  but  not  always,  provided  to  correct 
the  errors.  The  sensor  errors  arise  because  the  color 
matching  curves  corresponding  to  the  picture  tube  phosphor 
primaries  always  have  negative  values  over  some  portion  of 
the  wavelength  spectrum,  while  real  sensor  characteristics 
have  only  nonnegative  values.  In  theory,  ideal  sensors 
could  be  made  by  providing  a real  sensor  of  the  correct 
spectral  shape  for  each  negative  lobe  in  the  color  matching 
curves,  and  subtracting  its  output  from  that  of  the 
appropriate  positive  lobe  sensor  of  each  curve.  This  would 
require  eight  or  nine  sensors  and  is  never  done  in 
practice.  Instead,  three  sensors  are  used  whose  spectral 
characteristic  curves  approximate  the  main  positive  lobes 
of  the  color  matching  curves  of  the  receiver  primaries 
(11).  In  many  cases,  no  further  restoration  is  done,  and 
color  errors  are  present  owing  to  the  absence  of  the 
negative  sensor  lobes.  These  errors  usually  appear  as  a 
loss  of  color  saturation  at  the  receiver.  m some 


television  cameras,  a linear  matrix  operation  is  applied  to 
the  three  color  signals  in  order  to  better  approximate 
receiver  tristimulus  values.  The  matrix  typically  has 
negative  off-diagonal  elements,  which  approximate  the 
action  of  the  missing  negative  lobes  by  subtracting  a 
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fraction  of  each  of  the  other  signal  values  from  each 
signal.  The  exact  values  of  the  matrix  elements  are 
typically  chosen  by  minimizing  the  average  color  error,  as 
measured  by  one  of  the  "uniform"  color  difference  formulas, 
over  some  ten  or  twenty  test  color  samples  [12].  Signal 
matrixing  can  reduce  television  color  errors  due  to 
imperfect  sensor  characteristics  to  a very  small  average 
level  [11,12]. 


In  the  case  of  subtractive  color  imaging,  i.e. 

photography,  the  most  common  restoration  method  is 

photographic  masking.  Masking  can  be  described  as  the 
modification  of  one  image  by  the  information  contained  in 
anotm-r  [14,15].  Photographic  masking  is  intended 

primarily  to  correct  for  undesired  absorption  of  dyes.  A 
negative  image,  or  mask,  is  generated  for  each  case  of 
significant  cross-talk  among  the  three-color  images.  When 
these  masks,  up  to  six  in  number,  are  placed  in  exact 

registration  with  the  three  superimposed  original  cyan, 

t 

magenta,  and  yellow  images,  the  cross-talk  components  of 
the  original  dye  images  are  largely  cancelled  by  the  masks 
[14].  Masking  is  therefore  a similar  operation  to 
matrixing  in  color  television,  even  though  the  sources  of 
the  errors  are  different  in  the  two  cases.  In  the 
television  case,  a .natrix  with  negative  off-diagona] 

elements  operates  on  the  three  exposure  signals  in  order  to 

correct  for  imperfect  sensor  characteristics.  Photographic 
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masking  applies  a matrix  with  negative  off-diagonal 
elements  to  the  vector  of  film  layer  densities  (which  vary 

as  the  log  of  the  exposures)  in  order  to  correct  for 
undesired  dye  absorptions. 


Because  of  the  effort  required  in  preparing  and 

registering  the  masks,  masking  is  mainly  used  in  cases 

where  many  reproductions  are  required  from  a single 

original.  Masking  is  widely  used  in  photo-mechanical  color 

reproduction  (for  example,  photo-lithography).  In 

Photo-mechanical  reproduction,  masking  is  incorporated  into 

the  process  of  preparing  separation  negatives  for  each  of 

the  printing  dyes[16] . since  the  separations  are  often 

produced  by  a color  scanner,  the  masking  operation  is 

readily  achieved  by  mr-ipulation  of  the  scanner  electrical 
signals  [15,16]. 
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3.  Digital  Color  Restorati< 


Color  errors  which  are  generated  by  an  imperfect  color 
reproduction  system  can  be  corrected  digitally.  A sampled 
color  image  is  described  by  a 3-dimensional  array  of 
numbers,  usually  nxnx3,  where  there  are  n samples  in  each 
of  two  spatial  dimensions,  with  3 color  values  stored  at 
each  spatial  image  point.  When  a color  image  exists  in 
such  a sampled  form,  or  can  be  put  into  sampled  form,  the 
great  speed  and  flexibility  of  a digital  computer  can  be 
put  to  use  in  correcting  the  color  errors  introduced  by 
imperfections  in  the  sensors  and  storage  devices  which 
recorded  the  original  image.  In  contrast,  correction  of 
color  errors  by  photographic  means,  for  example  by 
photographic  masking,  is  a slow  and  complex  process.  The 
production  of  several  masking  transparencies  and  their 
exact  registration  is  required  for  each  image  which  is  to 
be  corrected.  The  correcting  power  of  masking  is  limited 
to  altering  the  density  of  each  transparency  layer  by  a 
weighted  sum  of  the  two  other  layer  densities  at  that 
point,  as  was  described  in  Chapter  2.  In  digital 
restoration,  however,  the  correcting  algorithms  may  be 
complex,  nonlinear,  spatially  varying,  and  may  have  spatial 
memory.  The  speed  of  a digital  computer  when  used  along 
with  a television  display  also  allows  a degree  of 
interactive  processing  which  is  not  possible  with 

photographic  restoration.  The  penalties  incurred  in 


digital  image  restoration  include  the  requirement  for  the 
necessary  equipment  and  loss  of  accuracy  associated  with 
sampled  data  processing. 

3.1  Vector  Space  Formulation  of  Color  Analysis 

In  order  to  perform  color  restoration  in  a digital 
computer,  the  necessary  functions  and  equations  must  be 
converted  to  discrete,  sampled-data  form.  in  this  form, 
the  common  linear  operations  of  color  measurement  and  color 
reproduction  become  matrix-vector  operations  instead  of 
integral  operations.  For  example,  an  integral  operation  on 
a spectral  waveform  C(X)  of  the  form 


/xu 
51 


(X)  C (A)  dX, 


(3-1) 


for  i-1,2,3  can  be  discretized  by  replacing  the  continuous 
variable  X with  a finite  set  of  points  XX  ...A  and 

lc  n 

forming  the  approximate  discrete  equation 


11 

ci  " l w S (X  ) C(X.) 
j=l  J 1 J 3 


where  w1'w2'---Wn  an  the  weighting  coefficients  of  the 
quadrature  integration  formula  used  (1].  By  defining  a 
vector  c whose  i th  component  is  C(A.),  and  a matrix  whose 
ij  th  component  is  w.S.fX.),  eq.  (3-2)  car  be  compactly 
stated  in  matrix  form  by 
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K, 
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(3-3) 


(3-5) 


Since  many  of  the  important  equations  of  colorimetry  and 
color  reproduction  are  of  the  form  of  eq.  (3-1),  matrix 
operators  and  equations  are  very  useful  in  the  analysis  of 
sampled  data  models  of  color  imaging  systems  and  in 
colorimetry  [2].  The  equations  of  colorimetry  are  strictly 
sampled  data  equations  in  any  case,  since  color  matching 
curves  are  obtained  by  experiments  with  human  observers  at 
a finite  number  of  optical  wavelengths  [3] . The  continuous 
color  matching  curves  which  are  sometimes  published  are 
obtained  by  interpolation  of  the  original  sample  values. 

The  utility  of  matrix  notation  can  be  demonstrated  by 
examining  color  matching  and  metamerism  [4].  Two 
spectrophotometr ic  distributions  of  light  are  said  to  be 
color  matched  if  they  cannot  be  distinguished  from  each 
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x = S c 
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r-  R<*2> 


(3-9) 


.R<Xn> 


with  R ( X ) representing  the  spectral  reflectance  of 
reflective  sample.  If  the  color  is  from  a reflects 
sample,  then 


c = Er 


(3-10) 


The  tristimulus  values  t for  a reflective  sample  are  given 


t = TEr 


(3-11) 


If  £E  and  r ^ represent  the  spectral  reflectances  of  a 

5 iil 

sample  and  a matching  sample  respectively,  then 


t = TErs  = TErf 


(3-12) 


— t£s  -£m!  " £ 


(3-13) 


Two  reflectivities  are  thus  said  to  be  metameric  with 
respect  to  an  illuminant  E and  a standard  observer  T when 
their  difference  vector  lies  in  the  null  space  of  the 
matrix  TE. 


3.2  Modelling  Color  Imaging  Systems 


In  order  to  achieve  accurate  color  restoration 
digitally,  an  accurate  vector  model  of  the  color  imaging 
system  is  necessary.  A general  model  of  an  imaging  system 
senses  and  stores  color  values  is  shown  schematically 
in  figure  (3-1).  At  each  image  point,  a spectral  power 
distribution,  represented  in  sample  form  by  the  vector  c of 
dimension  n,  serves  as  the  input  to  a set  of  sensitivity 
functions  represented  by  the  matrix  S,  with  outputs 
consisting  of  the  three  components  of  the  vector  x.  The 
elements  S,  C,  and  x are  defined  and  related  by  eqs. 
(3-3),  (3-4),  and  (3-5).  As  an  example,  the  rows  of  S 
might  be  samples  of  the  three  layer  sensitivities  of  a 
color  photographic  film.  In  general,  the  linear  operation 
modeled  by  the  matrix  S is  followed  by  a nonlinear  function 
of  x which  results  in  another  3-vector  w.  The  nonlinear 
operation  F might  represent  limiting  of  signals  in  a color 
television  system,  or  the  nonlinear  relationship  between 
layer  exposure  and  layer  density  in  color  photographic 
film.  The  three  components  of  w are  stored  and  are  at  some 
later  time  used  as  input  signals  to  a process  G,  generally 
nonlinear,  which  displays  the  signals  as  an  output  spectral 
distribution  c , again  represented  by  n samples. 

The  functional  relationship  between  the  input  spectral 
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distribution  c and  the  displayed  distribution  c0  for  this 
model  can  be  written  as 


£o  = GFSc  (3-15) 

where  G and  F are  general  vector  operations  of  appropriate 
dimensionality  and  are  nonlinear,  in  general.  It  is 
clearly  impossible  for  the  modeled  system  to  generate  a c 

O 

vector  equal  to  the  input  vector  c.  The  S operation  has 
reduced  the  dimensionality  of  £ from  n (usually  30,  40,  or 
80)  to  3,  with  an  irreversible  loss  of  information.  It  is 
possible,  in  theory,  to  generate  a c0  which  is  metameric  to 
c,  since  each  color  is  color imetrically  specified  by  three 
tristimulus  values.  For  a color imetr icall y perfect  color 
reproduction  system,  the  input  and  output  tristimulus 
vectors,  t and  t0  are  equal  at  each  image  point,  or 

— 0 = — 0 = TGFSC  - Tc  = t (3-16) 

This  relation  holds  if  the  following  two  statements  are 
true : 

1)  The  rows  of  S are  color  matching  curves.  If  this 
is  true,  then  x=Sc  is  a tristimulus  vector  of  c and 
may  be  easily  converted  to  the  tristimulus  vector 
corresponding  to  any  other  set  of  color  matching 
curves  (for  example  the  rows  of  T)  . 

2)  The  operator  G F is  an  inverse  tristimulus 
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operator;  that  is,  it  takes  a tristimulus  vector  x and 
generates  a waveform  c which  is  a solution  of  the 
linear  equation  x=Sc  . This  implies  that  G F is  a 
linear  operator  even  though  F is  generally  nonlinear. 

When  these  two  conditions  are  met,  the  color  at  each 
pixel  in  the  reproduced  image  will  match  color imetrically 
the  corresponding  point  in  the  input  image.  As  stated  in 
Chapter  2,  real  color  reproduction  systems  do  not  meet 
these  conditions  exactly.  Sensor  spectral  characteristics 
are  usually  only  approximations  to  a set  of  color  matching 
curves,  and  the  subsequent  recording  display  sequence 
operation  G F is  always  nonlinear  owing  to  the  limiting  of 
signals  and  other  no:ilinearities  in  the  storage  medium  or 
the  display  (5,  6]  . 

In  order  to  perform  digital  restoration  on  a digitized 
color  image,  the  foregoing  model  must  be  extended  to 
include  the  color  scanner  which  digitized  the  displayed 
image.  in  subsequent  sections,  the  special  case  of 
reversal  color  film  as  an  image  storage  medium  will  be 
examined.  To  perform  digital  processing  on  a 
photographically  recorded  transparency,  the  transparency 
must  be  scanned  by  a color  analyzing  system  which  outputs 
three  color  signal  values  at  each  point  of  a chosen  spatial 
grid  array.  The  completely  general  color  analyzing  system 
model  including  a scanner  is  shown  in  figure  (3-2),  with 
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the  film-scanner  special  case  specified  beneath  the  general 
system  component  blocks.  It  should  be  noted  that  while  all 
of  the  component  blocks  shown  are  present  in  a digitized 
film  color  system,  they  may  not  all  be  present  in  other 
digital  image  color  systems.  A digital  television  system, 
for  example,  might  not  display  and/or  store  the  images 
before  digitization. 

For  the  digital  color  analyzing  system  of  figure 
(3-2),  the  relation  between  the  input  color  vector  c and 
the  stored  observable  v is  given  by 

v = HGFSc  (3-17) 

where  H,  the  characteristic  function  of  the  scanner,  is  a 
nonlinear  function  of  c , but  may  be  linear  to  a good 
approximation  over  a wide  range  of  signal  levels  [7] . If 
the  signal  v were  used  to  drive  a display  television 
monitor,  a colorimetric  match  between  the  displayed  color 
and  the  input  color  c will  be  obtained  if  v is'  the 
tristimulus  vector  of  c in  the  system  defined  by  the 
television  display  primaries.  If  T is  the  matrix  of  color 
matching  curves  in  that  system,  then 

v = Tc  (3-18) 
or 
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for  a color imetrically  ideal  digital  color  system. 

3.3  The  Color  Estimation  Problem 

The  problem  to  be  solved  can  be  stated  as  follows:  How 
can  the  tristimulus  values  of  an  input  color  to  the  system 
of  figure  (3-2)  be  most  accurately  determined  when  the  two 
conditions  of  the  ideal  colorimetric  reproduction  system  do 
not  hold?  Cy  observing  the  digitized  signal  v(x,y)  over 
some  discrete  array  of  image  points  in  the  x and  y spatial 
dimensions,  it  is  desired  to  estimate  t(x,y),  the 
tristimulus  vector  at  each  image  point,  where  t is 
derivable  from  the  input  spectral  distribution  c by  the 
relation  t=Tc . 

The  estimation  problem  may  be  broken  into  two  parts, 
each  corresponding  to  one  of  the  conditions  of  ideal 
reproduction  which  are  violated  by  real  color  reproduction 
systems.  Correspondingly,  eg.  (3-17),  which  describes  the 

whole  system  sequence,  may  be  broken  into  two  serial 
operations 
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v = HGFx 


(3-21) 


The  estimation  process  proceeds  in  two  steps.  First,  an 
estimate  of  x is  determined  from  the  observable  v by 
inverting  eq.  (3-21).  Second  the  estimated  jc  is  used  to 
estimate  c by  inverting  eq.  (3-20)  . It  will  be  shown 

A 

later  that  the  desired  tristiinulus  estimate  t is  best 
estimated  by  taking  the  estimated  c and  letting  t=Tc . The 
two  requirements  of  ideal  colorimetric  reproduction  may  be 
related  to  the  above  equations  as  follows: 

1)  The  matrix  S in  eq.  (3-20)  must  have  rows 
consisting  of  color  matching  curves. 

2)  The  operation  HGF  in  eq.  (3-21)  must  be  linear  and 
invertible. 


In  an  idealized  case  where  condition  (2)  is  true,  x is 
obtained  from  eq.  (3-16)  by  simple  matrix  inversion 


x = (HGF)  v 


(3-22) 


If  condition  (1)  is  also  true,  then  x is  a tristimuli 


vector  of  c and  there  exists  a 3x3  matrix  A such  that 


T = AS 


(3-23) 


and,  therefore 


t = Tc  = ASc  = Ax 


(3-24) 


Pot  a teal,  non-ideal  system,  no  matrix  A satisfying  eg. 

(3-23)  exists  and  HGF  is  usually  invertible  only  over  some 
range  of  signal  values. 

In  summary,  it  is  desired  to  invert  the  set  of 
eguations 


x = Sc 


(3-25) 


V = HGFx 


(3-26) 


in  order  to  obtain  an  estimate  of  the  tristimulus  vector  t 

Which  is  optimal  in  some  as  yet  undefined  sense.  Equation 

(3-26)  is  non-linear,  deterministic  (if  observation  noise 

can  be  neglected)  and  is  exactly  determined  <x  and  v each 

are  three  dimensional).  Equation  (3-25)  is  linear, 

stochastic  (because  of  observation  noise  on  x and  possibly 

because  of  statistical  assumptions  which  may  be  made  about 

£>  and  is  underdetermined  (c  is  of  higher  dimension  than 

x) . The  two  step  estimation  method  is  diagrammed  in  figure 
(3-3) . 

Chapter  4 will  examine  methods  of  solution  of  eg. 
(3-25).  Chapter  5 will  examine  solution  methods  for  eg. 
(3-26),  with  emphasis  on  the  film-scanner  color  analysis 
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4.  Tristimulus  Estimation 

The  final  step  in  the  color  restoration  process  is  the 
estimation  of  the  tristimulus  values  at  each  image  point  in 
the  original  scene.  This  estimate  must  use  as  its  input 
auantities  signals  which  are  outputs  of  imperfect  spectral 
sensors,  and  which  are  usually  themselves  indirectly 
obtained  from  other  observable  quantities.  This  chapter  is 
concerned  with  the  estimation  of  tristimulus  values  from 
spectral  sensor  outputs  which  are  assumed  to  he  directly 
observable . 

In  vector  notation,  the  equations  to  be  solved  are 

x = Sc+e  (4-1) 

and 

t=Tc  (4-2) 

where 

S:  3xn  array  of  sensor  spectral  sensitivities 
T:  3xn  array  of  color  matching  curves 
c:  nxl  spectral  distribution  vector  of  input  color 
x:  3x1  vector  of  sensor  outputs 
e:  3x1  vector  of  observation  errors 
t:  3x1  vector  of  tristimulus  values  of  c 
The  problem  is  to  estimate  t when  S and  T are  known,  c and 
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e are  unknown,  and  x is  observed. 


4.1  Linear  Estimation  Methods 


Since  eo.  (4-1)  is  a linear  eauation  and  since  the 
techniques  of  linear  estimation  are  well  developed  [1,2,3], 
it  is  natural  to  attempt  to  estimate  t from  x using  linear 
estimation  methods.  In  order  to  optimize  any  estimator, 
however,  it  is  necessary  to  choose  a fidelity  criterion,  an 
expression  which  quantifies  the  color  difference  between 
any  two  sets  of  tristimulus  values,  t j and  t 2>  Such  a 

fidelity  criterion  should  ideally  satisfy  two  conflicting 
conditions: 

1)  The  fidelity  criterion  should  be  perceptually 
uniform,  i.e.,  the  error  value  given  by  the  fidelity 
criterion  should  be  proportional  to  the  perceived 
color  error  over  the  entire  tristimulus  space. 

2)  Maximization  of  the  fidelity  criterion  should  lead 
to  estimator  equations  which  are  solvable  in  closed 
form  and  with  a high  degree  of  generality. 

There  are  several  existing  color  distance  formulas 
which  are  perceptually  uniform  to  a good  approximation 
[7,8].  These  color  distance  formulas  (see  Section  2.1)  are 
complicated  functions  of  the  tristimulus  values  of  the 
colors,  and,  when  used  as  color  fidelity  criteria,  do  not 
satisfy  the  solvability  condition  stated  above.  in  order 
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to  obtain  solvable  equations,  it  is  necessary  to  use  a 
quadratic  error  criterion,  a type  of  error  criterion  whose 
use  is  almost  universal  in  linear  estimation  theory. 
Hence,  let  the  error  criteria  in  the  tristimulus  and 
spectral  distribution  spaces  be  the  quadratic  forms 

et  = “ i/  MCtj  - i2)]1/2  (4-3) 

and 

ec  = I<£i  - £2)T  -(%  ' £2)]1/2  (4-4) 

where  M and  N are  3x3  and  nxn  positive  definite  matrices, 
respectively.  Then  the  many  techniques  of  linear 
estimation  which  have  been  developed  over  the  years  may  be 
directly  applied  to  the  tristimulus  estimation  problem. 
These  error  criteria,  or  color  distance  formulas,  are  not 
perceptually  uniform,  but  they  do  satisfy  two  important 
conditions  which  a useful  color  distance  formula  should 
obey : 

1)  Reducinq  the  color  distance  between  two  colors  as 
determined  by  either  of  these  two  formulas  will,  in 
qeneral,  provide  an  improved  color  match  between  the 
two  colors. 

2)  When  the  color  distance  e,  or  e is  equal  to  zero. 

t c 

the  two  colors  are  perceptually  equivalent  under  a 
colorimetric  model.  The  converse  is  true  for  e^  but 
not  for  e . 


c 


In  the  remainder  of  this  chapter,  several  widely  used 
linear  estimation  methods  will  be  applied  to  the 
tristimulus  estimation  problem.  The  problem  will  first  be 
approached  under  a linear  repression  model.  Under  this 
model,  the  spectral  distribution  c is  assumed  to  be 
nonrandom  but  unknown.  No  statistical  information  on  c is 
assumed  available,  but  a priori  knowledge  in  the  form  of  a 
smoothness  constraint  is  utilized  to  improve  the  estimate. 
Least  squares  estimation  and  minimum  norm  methods  are  amonq 
the  techniaues  employed  to  solve  eas.  (4-1)  and  (4-2, 
under  the  regression  model.  The  tristimulus  estimation 
problem  will  also  be  approached  using  a stochastic  color 
model.  Under  this  model,  c is  assumed  to  be  a random 
vector  whose  first  two  moments  either  are  known  or  can  be 
approximated.  The  problem  can  then  be  solved  using  well 
known  Wiener  estimation  methods. 


4.2  Estimation  Under  a Linear  Regressi 


on  Model 


in  Chanter  3 it  was  shown  that  if  sensor  snectral 
sensitivities  can  be  expressed  as  linear  weighted  sums  of 
color  matching  curves,  then  tristimulus  values  can  be 
directlv  obtained  from  the  sensor  outputs  when  there  is  no 
observation  noise.  For  cases  in  which  this  condition  does 
not  hold,  a matrix  B can  be  found  which  aoproximates  the 
color  matching  matrix  T by  BS  in  the  least  souares  sense. 


\ 


L. 


Then  a tristimulus  estimator  ca..  be  obtained  by  letting 


A 

t 


BSc  = Bx 


(4-7) 


In  order  to  find  the  least  sauares  approximation  or 
fit  of  T by  S it  is  necessary  to  find  the  best  least 
squares  fit  of  each  row  of  T by  a weighted  sum  of  the  rows 
of  S.  Let 


and 


(4-8) 


(4-9) 


Then  it  is  desired  to  find  the  vectors  bi  for  i=l,2,j  which 
minimize  j|u  -Stb.  ||  where  the  n-dimensional  Euclidean  vector 


: 


Taking  the  transpose  of  each  side  and  observing  that  SS  is 


symmetric , 


b l = 4 ST  (SST)-1 


(4-12) 


Using  (4-12)  , (4-8)  , and  (4-9)  , 


B = T S^SSV1 


Substituting  this  expression  for  B into  eq 
the  least  squares  tristimulus  estimator 


a T T -1 

t = T S^SS1)  1 x 


(4-13) 


(4-7)  gives 


(4-14) 


The  matrix  product  S^SsV  is  defined  an  the 
pseudo-inverse  matrix  s+  of  the  3xn  matrix  S,  when  s has 
linearly  independent  rows  (21 . The  least  souares 
estimator,  eq.  (4-14)  is  therefore  of  the  form 


_t  = Tc=TS  x 


(4-17) 


A -f 


where  c=S  ± is  an  estimator  of  the  unknown  color  vector  c, 
obtained  by  solving  the  (underdetermined)  equation  x=Sc  for 

c using  s , the  pseudo-inverse  of  S.  Any  vector  c of  the 
form 


£ = S+  x + (I  - S+S)  v 


(4-18) 


where  v is  any  n vector,  is  also  a solution  of  x=Sc.  This 
may  be  seen  by  multiplying  each  side  of  (4-18)  by  s, 
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Y 


X 


(4-19) 


Sc  = SS+  x + (_S  - SS+S) 

since  SS+*I.  Of  all  the  solutions  given  by  ea.  (4-18), 
the  vector  c=S+x,  obtained  by  settinq  v=0  in  (4-18)  is  the 
solution  of  minimum  Euclidean  norm.  This  becomes  evident 
by  observinq  that  the  particular  solution  £ x is  orthoqonal 
to  the  homogeneous  solution  (I-S+S)v.  For  any  v 

l (I  “ i+S)  v]T  S+x  = VT(I  - :.+S.)  s X = 0 (4-20) 

Since  the  length  (norm)  of  the  sum  of  a fixed  vector  S x 
and  a vector  orthoqonal  to  S x is  minimized  when  the 
orthogonal  vector  is  of  zero  length,  c=S  x is  the  minimum 
(Euclidean)  norm  solution  of  x=Sc. 

The  particular  estimate  of  c given  by  c=S+x  is  only 
one  of  the  many  solutions  to  the  under determined  equation 
x=Sc.  Minimum  norm  estimation  provides  other  solutions 
which  may  be  better  in  the  sense  of  minimizing  a quadratic 
norm  given  by 

l|c| lN  = (CT  Nc)1/2  (4-21) 

The  nxn  matrix  N must  be  positive  definite  and  symmetric 
end  is  chosen  so  that  ||c  a measure  of  a property  of  c 
which  is  to  be  minimized.  Particular  choicer  of  N,  such  as 
those  leading  to  minimal  energy  or  maximal  smoothness  of  c, 
will  be  discussed  after  deriving  the  general  minimum  norm 
estimator . 
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(4-14)  is  obtained.  The  relationshiD  between  the  least 
squares  and  minimum  norm  solutions  for  general  N is  given 
in  the  following  cheorem  (5),  which  is  given  without  proof: 

Theorem  - Let  N be  positive  definite  and  S fce 
real.  Let  the  M-least  squares  minimum  N-norm 
inverse  of  A.  Then, 


(4-27) 


The  theorem  states  that  the  minimum  N norm  estimator  is 

equivalent  to  the  N 1 least  squares  estimator.  This  means 

that  if  tne  least  squares  estimator  eq.(4-14)  used  (cTff*c) 

• T 

as  its  n-space  norm  instead  of  (c  c)  , it  would  be 

identical  to  the  expression  of  eo.(4-26). 

The  choice  of  the  nxn  matrix  N in  the  N-normis 

determined  bv  the  particular  solution  which  is  to  be  chosen 
from  the  many  solutions  of  x=Sc.  if  n=I,  the  identity 

matrix,  the  norm  becomes 


which  is  equivalent  to  the  energy  of  c.  Another  approach 
in  restricting  the  estimate  of  c is  to  try  to  choose  N so 
as  to  incorporate  a priori  knowledge  of  c into  the 
estimate.  One  such  piece  of  a priori  knowledqe  is  that  the 
spectral  intensity  curve  which  c represents  is  "smooth"  in 
some  sense.  A useful  smoothness  criterion  is  that  of  the 
average  squared  second  difference  of  c (6).  If  this 


smoothness  criterion  is  used,  the  solution  to  x=Sc  which 
minimizes  the  quantity  c^c  is  sought,  where 


£ - = J2  KCl+1  " ^ " <Ci  ' Ci-1)]2  = J,  (cl+l  " 2cl  + ci-^; 


If  a vector  y=Dc  is  defined  by 


1 -2  1 0 3 ...  0^  rCl 

0 1 -2  1 0 ...  0 c2 

0 o 1 -2  1 ...  o • 

• • 

o - . 0 1-2  iJLc 


= Dc  (4-28) 


T . 


then  y y is  the  desired  average  squared  second  difference. 
Th  u s , 

£T  Ji£  = l y, 2 = £Tv  = cT  dtdc 
1=3  1 

and  the  nxn  matrix  n is  then 


0 0 


N = D D = 


6 -4 


(4-29) 


The  matrix  N thus  derived  cannot  be  directly  substituted 
into  eq . ( 4-26 ) , the  minimum  norm  estimator,  since  it  is 
singular  (each  row  and  column  sum  to  zero).  This  obstacle 
can  be  removed  by  replacing  N by  N + el  where  e is  a 
positive  constant,  small  compared  to  unity. 
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The  least  squares  and  minimum  norm  solutions  treated 
thus  far  are  deterministic  solutions  co  eqs.  (4-1)  and 
(4-2).  The  observation  noise  e in  eq.  (4-))  was  assumed 
to  be  negligible.  Tristimulus  estimation  under  a 
regression  model  will  now  be  extended  to  include  additive 
observation  noise  whose  first  two  moments  are  assumed 
known.  The  regression  model  is  given  by  the  set  of 


eauations 


x = Sc  + e 


( 4-30a) 


E(e)  = 0 


( 4-30b) 


E(ee  ) = R 


(4-30C) 


where  E denotes  the  ensemble  expectation  operator,  and  R 


ee 


is  therefore  the  3x3  symmetric,  positive  definite 
covariance  matrix  of  the  observation  error  e.  Equations 
(4-30)  are  a special  case  of  the  general  linear  regression 


problem.  What  is  found  is  a set  of  three  3-vectors  1 for 

—i 


i = l,2,3  such  that  1.x  is  an  estimator  of  t.  , the  i th 

~i  — i 


element  of  t.  Thus, 


- T T 

t.  = £ , x~u.  c = t . 

i —i  — —1  — i 


where  u.  is  the  i th  row  of  the  tristimulus  matrix  T. 


Clearly  it  would  be  desirable  that 


E(t ) = t 


1 

1 


<i 


63 


If  t satisfies  this  relationship  for  all  true  values  of  t 
the  estimator  is  said  to  be  unbiased.  For  an  unbiased  t 

E<Ai  *)  - U J c 

Now, 

E(Ai  *)  = A*  E(x)  = aJ  Sc 

So,  for  an  unbiased  estimator 

T T 

Ai  *£  = uj  c 

for  all  c,  or 

- -1  = (4-31) 

If  this  condition  holds,  then  t. -ufo  is  said  to  be 
unbiasedly  estimable. 

Unfortunately,  the  quantity  t.  is  not  unbiasedly 
estimable  in  the  case  of  tristimulus  estimation.  The 
condition  of  eq.  (4-31)  does  not  hold  in  general,  since  it 
requires  that  the  color-matching  curve  us  be  expressible  as 
a weighted  sum  of  the  three  rows  of  s (which  are  the  three 
sensor  soectral  sensitivities).  Note  that  this  condition 
is  equivalent  to  the  condition  of  eq.  (3-23).  since  the 
dimensionality  n of  the  rows  of  s is  far  greater  than 
three,  eq . (4-31)  cannot  hold  for  general  s.  (in  equivalent 
description  is  that  unbiased  estimation  is  not  possible 
when  the  regression  model  is  underdetermined . 
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Although  an  unbiased  estimator  is  not  obtainable, 


it 


h 


k 

i 

i 


is  possible  to  seek  a minimum  bias  estimator.  A linear 

T T 

function  1.  x is  said  to  be  a minimum  bias  estimator  of  u,  c 


i | T i 

1 T 1 1 

S ft,  - u. 

= inf 

S ft  - u. 

1 1 — —1  —i  1 

0 

Minimum  bias  estimators  are  not  unique,  in  general.  It  is 
desired,  then,  to  find  an  estimator  which  has  least 
variance  in  the  class  of  linear  minimum  bias  estimators. 

This  estimator  is  called  the  best  linear  minimum  bias 
estimator  ( BLMBE ) . The  variance  of  a scalar  random 
variable  a is  defined  to  oe 

V(a)  = E(a  - E(a))2 

U 

T T 

Let  1.x  be  an  estimator  of  u.  c . Then, 

F.(ftJ  X - c)  = (ftT  S - uj)  c (4-32) 

is  the  bias  and  the  variance  is 

V(ftJ  *)  = Ai  Age  Aj.  (4-3  3) 

T 

The  problem  is  to  minimize  1^ in  the  class  of  1 for 
which  US  1 -u  l|  is  a minimum.  The  norm  in  the  three 

_ _i  — i 

dimensional  space  is  defined  to  be  the  Euclidean  norm 

Mill  ■ »V/2 

T 1 /2 

Let  the  norm  in  R be  defined  by  ||x||=(x  Mx)  where  M is 
positive  definite.  The  answer  is  given  by  the  minimum 
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I-norm  M-least  souares  solution  of  the  inconsi 


stent 


equation  S xi  = u 


-i  = (/)+: 


The  estimator  of  u c is 

— i — 


-i 


(4-34) 


using  the  duality  result  of  eg. (4-27*.  The  variance  of  the 
estimator,  assuming  uncorrelated  observation  error,  is 


v(ATx)  - AT  A - a2  ATA  - o,2 


(4-35) 


where 


An  explicit  expression  for  the  BLMBE  for  ur.'orrelated 

observation  noise  is  given  by 

* T T T -1 

ci  = ^ (SMS  ) 1 x 

The  best  linear  minimum  bias  tristimulus  vector  estimator 
is  then 


t - TMST  (SMS1)-1  X 


(4-36) 


r t 

I L 


A A 

C2 


It  can  be  seen  that,  if  M=I,  the  DLMBE  is  eruivalent 
to  the  least  squares  estimator  of  ea.  (4-14).  If  M=N_1, 
the  BLMBE  is  eauivalent  to  the  minimum  norm  estimator  of 
eq.  (4-26)  . 


In  the  event  that  the  observation  noise  is  correlated, 
the  reqression  model  can  be  extended  to  account  for  it. 


E(f ) = 0 


( 4 — 3 7 a ) 


E(eeT)  = a2  W 


( 4-37b) 


where  W is  non-sinqular  and  positive  definite  and  a is 
unknown.  Since  W is  non-sinqular  and  positive  definite,  H 
exists  such  that 


-1  T 2 
W = H H = H 


(4-38) 


z = Hx 


e = He 
S = HS 


' 


Mi 


Then 


(4-39a) 


Ee  i = 0 


( 4 — 3 9 b ) 


r.  T 2 
Ee^  = a I 


(4-40) 


The  model  defined  bv  eas  f4-')Q\  an j ».«  . 

y eas*^  and  (4-40)  has  uncorrelated 


noise,  and  can  be  solved  as  before.  The  estimator  of  t is 


given  again  by  eg.  (4-36)  where  S is  replaced  by  s1(  and  x 


is  replaced  by  z.  Thus, 


1 - TMS[  (Sj  M S')'1  z 


1 HT(HSHST  HT)-1  Hx  - TMST  (SMST)-1  X 


(4-41) 


The  resulting  estimator  of  eg. (4-41)  is  identical  to  the 
uncorrelated  noise  estimator  of  eg. (4-36).  The  estimator's 
independence  of  ‘he  noise  correlation  is  a result  of  the 


fact  that  the  equation  being  solved  is  underdetermined,  it 
is  of  interest  to  look  at  the  overdetermined  and  exactly 
determined  cases.  if  the  general  solution  to  the 
correlated  noise  problem  eg.  (4-37)  and  (4-33)  is  given  by 


I 


A / , *f 

£ = (HS,  Hx  (4-4  2) 

where,  again,  (HS)+  is  the  Moor  e-Penrose  pseudo-inverse  of 
HS  (3] . For  the  overdetermined  case,  the  pseudo-inverse  is 
defined  by 


A+  ,aTa,-1  .T 

A = (A  A)  A (4-43) 

where  A is  mxn  with  m>n  and  rank  (A)=n.  In  the 
underdetermined  case,  the  pseudo-inverse  is  defined  by 


A+  = 


aW)'1 


(4-44) 


where  A is  mxn  with  m<n  and  rank  (A)=m.  In  the  exactly 
determined  case,  rr.*n  and  !*  =a"\  Substituting  these 
expressions  into  eq.(4-42)  in  turn  gives  for  the 
overdetermined  case 


c = (.',T  W 1S)  1 S1  W"1  x 


(4-45) 


In  the  underdetermined  case 


c = ST(SST)  1 x 


(4-46) 


In  the  exactly  determined  case 


c = S ^ x 


(4-47) 


The  best  linear  estimator  is  independent  of  the  correlation 
properties  oi  the  noise  when  the  system  to  be  solved  is 
underdetermined  or  exactly  determined. 
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In  the  tristimulus  estimation  problem,  which  is 
underdetermined,  the  linear  estimators  which  have  been 
discussed  thus  far  (least  squares,  minimum  norm,  and  BLMBE) 
are  all  of  the  general  form 


t = Tc  = TS~  x 


(4-48) 


where  S is  a generalized  inverse  of  S.  When  the  vector 
3 n 

norms  in  R and  R are  taken  to  be  Euclidean  norms  (i.e., 
M=N=I) , the  different  generalized  inverses  all  become  the 
pseudo-inverse,  and  all  three  estimators  are  identical 


| = Tc  = TS + x = IST(SST)_1  x 


(4-49) 


4.3  Stochastic  Color  Estimation 


The  regression  model  of  the  previous  section  included 
additive  observation  noise  whose  first  two  statistical 
moments  were  assumed  known.  The  SDectral  distribution 
vector  c of  the  input  color  was  assumed  to  be  unknown  but 
nonrandom;  no  statistical  assumptions  on  c were  included  in 
the  estimation  model.  In  the  following  discussion,  the 
model  is  altered  to  include  a random  c vector.  The  random 
color  model  is  given  by 


x = Sc  + e 


(4-50) 


t = Tc 


(4-51) 


BT- 


where,  again,  S and  T are  known  3xn  matrices,  x is  the 
observable  3-vector  and  t is  the  tristimulus  3-vector  which 
is  to  be  estimated.  The  statistical  properties  of  the 
unknown  n— vector  c and  3— vector  e are  given  by 


Ec  = m 
— — c 

( 4-52a) 

E[  (c  - m ) (c  - 
— — c — 

m )T]=  R 

—c  — cc 

( 4-52b) 

Ee  = 0 

( 4-53a) 

E(eeT)  = R 
— — ee 

( 4-53b) 

ECe^  0=0 

(4-53c) 

j = 1 , 2 , . . . , n . 

A linear  estimator 

of  c of 

the  form 


c = a + Ax 


(4-54) 


will  now  be  sought  for  which  a is  a real  vector  and  A is  a 
real  matrix  chosen  so  that 


^ A E[(c  - c)  (c  - c)T] 


(4-55) 


is  minimized.  The  minimization  of  0 is  defined  to  mean 

“ c 

that  for  any  other  linear  estimator  c*  and  its 

corresponding  O',  the  matrix  Q'-O  is  positive 

~c  — c — c 


semi-definite.  When  Qc  is  minimized,  the  expected  sauared 


' 

errors  of  the  components  of  c (the  diaqonal  elements  of  Q ) 
are  each  minimized.  This  insures  that 

e2  A F[(c  - c)T  (c  - c)] 

~ ~ ~ - 1 (4-57) 

tie  sum  of  the  squared  component  errors  of  c,  is  also 

minimized.  Most  importantly,  the  minimization  of  0 also 

- c 

results  in  the  minimization  of  the  matrix  of  tristinulus 
errors  gt  when  a tristimulus  estimator  t is  derived  from  c 
by  lettinq  t=Tc.  Let 


St  A Ef  = E[T(c-c)(c-c)T  TT]  = IS/  (4-53) 

If  0 c - 0C  is  positive  semi-definite,  then 

H I - 0 = T (o’  - n ) tt 

t -*-t ‘C  — c — 

is  also  positive  semi-definite  since  the  rows  of  T are 

linearly  independent,  and  Q is  therefore  minimized  when  0 

~c 

is  minimized.  The  spectral  distribution  estimator  which  is 

[sought  has  the  form  of  eq.  (4-54)  and  minimizes  eq. 

(4-55).  The  solution  is  the  well  known  discrete  Wiener 
estimator  (see  reference  [3]  for  a derivation),  given  by 


jk 


where 


R^x  = E[(c  - ffl.Xx  - iO  ] 


( 4-61a) 


Rxx  = EKx-jiHx-^)  ] 


(4-61b) 


m = Sm 
— x — c 


(4-61c) 


Substituting  eqs.  (4-59)  and  (4-60)  into  eq.  (4-54)  gives 


c = m + R R (x-m) 
— -c  cx  ~xx  — — x 


(4-62) 


By  manipulating  eqs. (4-50)  and  (4-61), the  necessary 

correlation  matrices  are  found  to  be 

R = E[  (c  - m ) (x  - m )T]  = R ST 
— CX  — — C — “X  -cc  — 


R = E [ (x  - m )(x  - m )T]  = S R ST  + R 
—xx  — —x  — —x  cc—  —ee 


Substituting  these  last  two  relations  into  (4-62)  gives 
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- ' =c  + 4c  *T  <S  iT  + 

The  corresponding  wiener  tristimulus  estimator,  t-Tc,  i, 
given  by 


i = T(m  + R ST  (s  R ST  + R \ 

C -cc  - v-  Arc  - *ee-)  (2i  ~ 


— —x 


(4-63) 


The  color  covariance  matrix  Rcc  might  reasonably  be 

approximated  by  a first  order  Markov  covariance  matrix  of 
the  form 


R 

c c 


.1  p 

p 1 

2 

P P 


n-1 

P n 


L "n-1 


(4-64) 


where  p,  the  correlation  coefficient,  is  a positive 
constant  less  than  unity. 

4.4  Tristimulus  Estimation  Results 

The  tristimulus  estimators  which  have  been  discussed 
in  the  foregoing  sections  have  been  tested  by  means  of  a 
computer  simulation  which  uses  ten  test  colors  and  also  by 
using  a computer  simulation  which  incorporates  real 
multispectral  images.  The  set  of  ten  test  colors  consisted 
of  measured  reflectivities  (reference  4-9)  of  natural  and 
man-made  objects  which  are  typical  of  those  which  might 
occur  in  a scene  to  be  reproduced.  The  ten  spectral 
reflectivities  are  shown  in  figure  (4-1).  They  correspond 
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*7* 


tc  the  following  objects: 


1)  flesh 

2)  grass 

3)  sky  blue 

4)  red  brick 

5)  purple  flower 

6)  blue  dye 

7)  green  dye 

8)  yellow  dye 

9)  red  dye 


10)  flat  neutral  gray 


These  reflectivities  were  multiplied  at  each  wavelength  by 
a daylight  illuminant  function  (CIE  Illuminant  C)  to 
generate  the  ten  input  test  colors.  Each  reflectivity 
curve  represents  a measured  reflectivity,  except  (3)  and 
(4).  Number  (3)  is  a curve  which,  when  multiplied  by 
Illuminant  C,  gives  a typical  spectral  energy  curve  of 
light  from  the  sky.  Number  (4)  is  a hypothetical  neutral 


reflector  whose  reflectivity  is  0.5  at  all  wavelengths. 

The  spectral  energy  curve  of  CIE  Illuminant  C is  shown  in 
figure  (2-3). 

The  spectral  sensitivity  characteristics  which  were 
incorporated  into  the  model  were  of  several  kinds.  An 
important  set,  representing  a typical  reversal  color  film 
and  lens  combination,  is  shown  in  figure  (4-3).  These 
curves  are  the  spectral  products  of  the  film  layer  taking 
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sensitivities,  fiqure  (4-la)  and  the  lens  optics 
transmissivity,  fiqure  (4-4b) . The  80  samples  of  each  of 
tite  3 curves  of  fiqure  (4-3)  become  the  elements  of  the 
3x80  sensitivity  matrix  S.  The  colorimetric  quality 
factors  o for  the  red,  green,  and  blue  sensor  curves  of  s 
are,  respectively,  0.414,  0.822,  and  0.983.  The  green  and 
especially,  the  blue  sensors  of  this  sensor  set  are  very 
good  approximations  to  color  matching  curves.  The  red 
sensor,  by  contrast,  is  not  close  to  any  color  matching 
curve  primarily  because  it  peaxs  at  about  650  nanometers,  a 
wavelength  which  is  about  50  nm.  longer  than  the  neak 
wavelength  of  any  ideal  color  matching  red  sensor.  The 
difference  can  be  seen  by  comparing  the  sensor 

characteristics  S,  figure  (4-3),  with  the  set  of  color 
matching  curves  which  are  nearest  to  S in  the  least  squares 
sense,  shown  in  figure  (4-5).  The  green  sensor  of  s is 
also  shifted  slightly  toward  the  longer  wavelengths 
relative  to  its  ideal  colorimetric  position,  but  not  so 
much  as  the  red  sensor.  The  reason  for  the  exaggerated 
spread  of  peak  sensitivities  in  the  color  film  is,  as 
described  in  Chapter  2,  to  increase  color  saturation  for  a 
more  pleasing  (but  less  accurate)  color  rendition. 

The  linear  estimators  of  the  input  waveform  c can  t a 
graphically  compared  using  the  film  sensor  s.  The 

estimator  are  of  the  form 


l 


TRANSMISSIVITY 


1 


I 


c = c + S x 

— -- o — — 


where  S is  a generalize  inverse  of  S and  c0  is  the  zero 
vector  except  in  the  Wiener  case.  The  generalized  inverses 
are  one  of  the  following: 


1)  pseudo-inverse 


s+  = sT(ssT)  1 


2)  constrained  inverse: 


-1  -1  T -1 

S = N S (SN  S ) 


3)  Wiener  inverse: 


sT  = mst(smst)  1 

— W 


where  N is  the  smoothing  matrix  defined  in  ea.  (4-29)  , and 
M is  the  Markov  covariance  matrix  given  by  ea.  (4-64)  . 
Since  S is  a 3x80  matr.’x,  each  S is  an  80x3  matrix  whose 


columns  may  be  graphed  as  the  rows  of  S were  graphed  in 


figure  (4-3).  The  columns  of  S , the  pseudo-inverse  of  S, 


are  shown  in  figure  (4-6).  The  columns  of  S and  S are 

— c ~w 


shown  in  figures  (4-7)  and  (4-8).  Figure  (4-8a)  shows  the 
Wiener  inverse  using  a correlation  coefficient  p=0.5  and 


The 


figure  ( 4 — 8 b ) shows  the  Wiener  inverse  with  p=0.9. 
estimate  c of  the  input  spectral  waveform  are  weighted  sums 
of  the  columns  of  the  appropriate  S (summed  with  3 mean 
vector  in  the  Wiener  case) . 
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Examples  of  the  waveforms  generated  by  the  estimators 
are  shown  in  figures  (4-9),  (4-10),  and  (4-11).  The  solid 
curve  of  figure  (4-9)  shows  a test  color  (sky  blue) 
obtained  as  the  product  of  the  illuminant  of  figure  (4-2) 
and  a "ref lectivity"  given  by  curve  number  3 of  figure 
(4-la) . The  dotted  line  in  the  figure  is  the 
pseudo-inverse  estimator  of  c given  by  c=S+x  where,  as 
usual,  x=Sc.  Figure  (4-10)  shows  the  constrained  smoothing 
estimate  c^^x  compared  with  the  same  input  test  color. 
Figure  (4-11)  shows  the  corresponding  estimate  of  c for  the 
case  of  the  Wiener  estimate  (P*0.9).  The  estimator  is 

£ = + MST (SMS7)"1  (x  - Sm  ) 

where  the  mean  vector  m was  taken  to  be  0.3  T(X, . The 
function  i(A)  is  the  illuminant,  figure  (2-3),  and  0.3 
corresponds  to  a somewhat  arbitrary  average  reflectivity. 
Of  these  three  solutions,  the  one  which  imposes  the  most 
smoothing  on  the  solution,  the  Wiener  estimate  with  p=0.9, 
is  clearly  closest  to  the  original  waveform. 

The  performance  of  the  tristimulus  estimators  is  shown 
in  Table  (4-1)  for  each  of  the  ten  test  colors.  These 
results  are  summarized  in  Table  (4-2)  by  averaging  the  data 
over  the  ten  test  colors.  Color  errors  are  described  three 
ways:  luminance  error  (where  luminance  = 1.0  corresponds  to 
the  illuminant  against  unit  reflectivity),  chrominance 
error  (Euclidean  distance  in  UCS  chromaticity  space),  and 
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WAVELENGTH , nm 

Figure  (4-9)  Pseudo-inverse  reconstruction  of  color  3 


WAVELENGTH,  nm 


Figure  (4-10)  Smoothing  inverse  reconstruction  of  color  3 


d 


WAVELENGTH,  nr 

Figure  (4-11)  Wiener  inverse  reconstruction  of  color  3 


WAVELENGTH,  nm 

Figure  (4-12)  Colorimetric  quality  factor  vs  wavelength, 

narrowband  sensor 


Table  (4-1} 

Tristimulus  Estimator  Performance  Against  Ten  Test  Colors 
Film  Sensitivity  Characteristics 


Color  Number 

1 

2 

3 

4 5 

6 

7 

8 9 

10 

True  Luminance 

.33 

.12 

.47 

.14  .18 

.09 

.23 

.44  .11 

.50 

Chromaticity  u 

. 24 

.19 

.18 

.28  .25 

.15 

.14 

.21  .38 

.20 

Chromaticity  v 

.32 

.33 

.29 

.34  .27 

.23 

.35 

.36  .32 

.31 

Pseudo-inverse 

Luminance  Error 

-.05- 

-.02- 

.07- 

■ . 02- . 02- 

.01- 

.04- 

■ . 07-  . 02- 

-.07 

Chrominance  Error 

.01 

.01 

.02 

.01  .02 

.04 

.03 

.01  .04 

.01 

Cube  Root  Error 

4.7 

3.3 

7.4 

3.0  3.7 

9.2 

5.7 

5.0  4.8 

5.8 

Smoothed  Pseudo-inverse 

Luminance  Error 

-.02- 

.01- 

.03- 

-.00-. 00- 

.01- 

.01- 

.02-. 01- 

■.02 

Chrominance  Error 

.01 

.01 

.01 

.00  .02 

.03 

.03 

.02  .03 

.01 

Cube  Root  Error 

2.1 

1.3 

3.3 

0.6  2.2 

5.4 

3.8 

2.6  2.7 

2.3 

Wiener , p =0.9 ,c=0. 

31 

Luminance  Error 

-.01- 

.01- 

.00- 

■.00-. 01- 

.00- 

.00- 

.01-. 01- 

.00 

Chrominance  Error 

.01 

.00 

.00 

.01  .01 

.01 

.02 

.01  .03 

.00 

Cube  Root  Error 

1.0 

0.3 

0.4 

0.6  1.7 

0.9 

2.1 

1.8  2.7 

0.2 

Table  (4-2) 


Tristimulus  Estimator  Performance  Averaged  Over  Ten  Test 
Colors,  Film  Sensitivity  Characteristics 


Est imator 
Type 

RMS  Error 
Luminance 

RMS  Error 
Chrominance 

RMS  Error 
Cube  Root 
Space 

Pseudo-inverse 

0.044 

0.022 

5.29 

Smoothed 
Pseudo- inverse 

0.014 

0.019 

2.78 

Wiener 

0.005 

0.011 

1.35 

Table 

(4-3) 

Tristimulus  Estimator  Performance  Averaged  Over  Ten  Test 
Colors,  Narrowband  Sensitivity  Characteristics 

Estimator 

Type 

RMS  Error 
Luminance 

RMS  Error 
Chrominance 

RMS  Error 
Cube  Root 
Space 

Pseudo-inverse 

0.272 

0.056 

39.82 

Smoothed 
Pseudo- inverse 

0.211 

0.054 

28.37 

Wiener 

0.037 

0.040 

10.67 

cube  root  uniform  perceived  error  (Euclidean  distance  in 
Lab  cube  root  coordinate  space). 

From  Table  (4-2)  , it  is  apparent  that  smoothing  of  c 
reduces  the  color  estimation  errors,  especially  the 
luminance  errors.  The  minimum  norm  property  of  the 
pseudo-inverse  generates  a waveform  possessing  the  least 
energy  (sum  of  the  squared  waveform  samples)  of  all 
solutions  to  x=Sc.  Therefore,  positive  weighted  summations 
over  the  estimated  waveform  tend  to  be  smaller  than 
equivalent  summations  over  the  “true”  input  waveform.  For 
this  reason,  pseudo-inverse  estimates  of  tristimulus  values 
tend  to  be  on  the  low  side.  Note  in  Table  (4-1)  that 
luminance  errors  using  the  pseudo-inverse  estimator  are 
uniformly  negative  for  all  ten  test  colors.  Imposing 
smoothing  on  c by  using  Wiener  estimation  generates 
waveforms  whose  luminance  errors  are  still  more  negative 
than  positive  (Table  4-1)  , but  whose  RMS  average  luminance 
error  is  reduced  over  the  pseudo-inverse  RMS  luminance 
error  by  a factor  of  nine  (Table  4-2)  and  whose  RMS 
chrominances  error  is  also  improved  by  a factor  of  two. 
The  net  error  improvement  in  the  perceptually  uniform  cube 
root  space  is  by  a factor  of  about  four. 

The  tristimulus  estimators  were  also  tested  using  the 
same  test  color  set,  but  using  sets  of  sensor 
characteristics  which  are  poorer  approximations  to  color 
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matching  curves  than  are  the  film  sensitivities  just 
discussed.  A set  of  narrowband  sensitivities  of  unit 
amplitude  peaking  at  wavelengths  of  450,  550  and  650 

nanometers  were  tested.  The  colorimetric  quality  factors 
of  these  sensors  are  0.114,  0.079  and  0.008.  These 

arbitrarily  chosen  sensors  are  not  the  best  narrowband 
sensors  color imetrically.  Figure  (4-12)  shows  a graph  of 

the  colorimetric  quality  q for  a narrow  band  filter  as  a 
function  of  its  center  wavelength.  The  optimal  narrowband 
wavelengths  are  approximately  445,  540  and  605  nanometers. 
The  average  color  errors  resulting  from  application  of  the 
tristimulus  estimators  using  the  narrowband  sensors  at  450, 
550  and  650  nanometers  are  summarized  in  Table  (4-3). 
Again,  the  Wiener  estimator  provides  the  best  tristimulus 
estimation  accuracy,  although  the  errors  are  larger  for 
each  estimator  type  than  they  were  using  the  film 

sensitivities.  The  improvement  provided  by  smoothing  when 
narrowband  sensors  are  used  is  apparent  when  the 
pseudo-inverse  and  wiener  estimates  of  color  number  three 
are  compared.  Figure  (4-13)  shows  the  pseudo-inverse 
reconstruction  of  color  three  using  the  narrowband  filters. 
It  consists  of  samples  of  the  input  waveform  at  the 
narrowband  wavelengths.  Figure  (4-14)  shows  the 
corresponding  Wiener  estimate  (P-0.9).  The  waveform  values 
at  the  filter  wavelengths  are  again  correct  (as  they  have 
to  be  if  x=Sc  and  the  filters  are  narrowband)  but  the 


90 


values  at  other  wavelengths  are  much  more  accurate  because 
of  smoothing.  Although  tristimulus  errors  are  much 
improved  because  of  smoothing  (see  luminance  error  in  Table 
(4-3)),  chrominance  errors  are  improved  by  a smaller 
factor.  This  is  because  chromaticities  are  ratios  of 
tr istimulus  values,  and  the  negative  tristimulus  errors 
tend  to  cancel. 

The  tristimulus  estimators  were  tested  using  a real 
six  band  multispectral  image.  The  six  Samples  at  each 
image  point  constituted  the  elements  of  c,  the  "true"  input 
spectral  waveform.  The  elements  of  the  3 x 6 array  S could 
then  be  selected  to  simulate  any  arbitrary  set  of  three 
taking  sensitivities.  The  x vector  generated  at  each  image 
point  by  x=Sc  served  as  an  input  vector  to  one  of  the 

estimator  algor ithms,  generating  an  estimated  image  to  be 
displayed  . 

The  input  multispectral  image  was  generated  by 
Photographing  a table  top  still  life  of  colored  objects 
through  six  narrowband  interference  filters.  The  filter 
center  wavelengths  were  440,  480,  520,  560,  600  and  640 
nanometers  and  their  half  power  bandwidths  were  about  10 
nanometers.  Each  filter  was  an  element  of  a sixteen  filter 
set  made  by  Oriel  corporation.  The  spectral  passbands  of 
the  set  are  shown  in  figure  (6-2).  The  exposures  were  made 
using  Kodak  Tri-x  film  in  a 35mm  camera  with  incandescent 
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illumination.  Exoosure  times  were  of  the  order  of  ten 
seconds  in  lenqth,  established  by  trial  and  error.  Care 
was  taken  that  the  field  of  view  of  interest  was  less  than 
twenty  degrees  in  order  that  the  interference  filters  were 
not  detuned  significantly  by  off-axis  rays  at  the  edge  of 
the  field.  The  six  negatives  were  digitized  using  a 
flatbed  microdensitometer  and  digitally  converted  to 
positive  images.  Photographs  of  the  six  digital  images, 
displayed  on  a cathode  ray  tube,  are  shown  in  figure 
(4-15)  . 

After  scaling  to  correspind  to  the  spectral  weighting 
of  the  illumination,  the  six  imaoes  were  weighted  and 
summed  according  to  the  color  matching  curves  of  a set  of 
television  display  primaries.  A photograph  of  the 
resilting  "true  color"  image  is  shown  in  figure  (4-16) (see 
color  plate) . The  six  images  were  also  weiqhted  by  factors 
simulating  sets  of  sensors  far  removed  from  ideal  color 
matching  curves.  One  such  set  consisted  of  narrowband 

! 

sensors  at  440,  520,  and  600  nanometers,  weiqhted  by  0.25, 

I 

1.0,  and  0.5,  respectively.  The  resulting  green  biased 
image  is  shown  in  figure  (4-17).  Operating  on  this  imaqe 
with  the  pseudo- i nver se  tristimulus  estimator  qives  the 
image  shown  in  figure  (4-18).  The  green  bias  is  removed, 
but  the  imaqe  is  dark  owinq  to  the  tendency  of  the 
pseudo- inverse  estimator  to  q nerate  low  tristimulus  values 
when  the  sensors  are  narrowband.  The  result  of  smoothing 
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Figure  (4-15)  Six  multispectral  images 
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is  shown  in 

the  Wiener 

estimated  imaqe  of 

figure  ( 

The  image 

is  bright 

and  color  balanced. 

The  main 

error  with 

respect 

to  the 

true 

color 

image 

desaturation  of  the 

br  ight 

reds , 

possiby 

because 

absence  of 

the  deep  red  640  nm. 

component 

in  the 

image . 

The  Wiener  estimator  performs  well  with  sensors  which 
may  not  be  color  balanced  and  which  may  be  eithei 
narrowband  or  broadband,  as  long  as  the  three  sensors  are 
near  the  ideal  values  of  445,  540  and  605  nm.  A more 
severe  test  takes  place  when  large  portions  of  the  visible 
region  are  not  covered.  Figure  (4-20)  shows  the  result 
when  the  blue,  green  and  red  display  primaries  are  driven 
by  equal  amplitude  narrowband  sensors  at  three  lonq 
wavelengths,  560,  600  and  640  nm.  Since  only  the  green  to 
red  components  are  present  at  the  input,  the  color 
distortion  is  great.  Reds  map  into  orange,  greens  become 
blue,  blues  become  black,  and  yellows  are  white.  With  this 
as  an  input,  the  Wiener  estimated  image  is  as  shown  in 
figure  (4-21) . The  primary  improvement  is  in  the  rendition 
of  th°  reds.  Blues  also  are  now  no  longer  black,  because 
of  the  smoothing  induced  blue  content  in  the  estimated 
spectra.  On  the  debit  side,  oranges  and  yellows  are  still 
rendered  as  white,  and  there  is  a slight  overall  pink  bias 
to  the  imaqe.  The  overall  color  improvement  achieved  by 
using  the  Wiener  estimator  can  be  judqed  by  examining  the 
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color  wheel  on  the  book  cover  of  each  of  fiqures  (4-19)  , 
(4-20 ) , and  (4-21) . 
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television  camera  tubes  and  cathode  ray  tubes  [1]. 
input  and  output  signals  are  called  vt  and 
respectively,  the  functional  relationshiD  is  niven  by 


If 
Vo  , 


7 

= V i 


where  the  constant  7 typically  has  a value  between  0.5  and 
2.0.  When  the  value  of  7 is  known,  and  other  operations 
have  not  also  taken  place,  the  qamma  nonlinearity  can  be 
corrected  by  raising  v.  to  the  power  l/7 . This  may  be 
easily  accomplished  when  signals  are  digitally  stored  in  a 
computer.  Another  important  invertible  nonlinearity  is  the 
density  vs.  exposure  relationship  in  photographic  film. 
Over  some  range  of  exposure  (radiant  energy  per  unit  area 
at  the  film  plane,  spectrally  weighted  by  the  film 
sensitivity  characteristic),  the  relation  between  film 
layer  density  and  exposure  is  very  nearly  logarithmic.  As 
will  be  shown  in  a later  section  of  this  chapter,  the 
relations  among  exposure  x,  density  d,  and  transmissivity  r 
for  a layer  of  reversal  color  transparency  film  can  be 
expressed  in  approximate  form  as 


d = b - y 1°8  x 


(5-1) 


and 


t = 10 


-d 


(5-2) 


where  b and  7 are  constants.  Substituting  (5-2)  into  (5-1) 
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1 


qives 


T = i0“(b-Y  loR  *>  = 10-b  vY 


(5-3) 


From  this  result,  it  is  clear  that  when  7 differs  from 
unity,  there  is  a cower  law  nonlinearity  between  the  input 
exposure  x and  the  output  transmissivity. 

2)  Non-inver tible  nonlinearities  - Nonlinearities  in 
imaging  systems  which  cannot  be  inverted  are  usually  caused 
b'  s*gnal  limiting.  Signals  whose  values  are  too  large  or 
too  small  for  the  dynamic  range  of  the  system  are  mapped 
into  the  upper  or  lower  clipping  levels  of  one  of  the 
system  components.  Examples  of  non-inver tible 
nonlinearities  include  the  clipping  action  of  the 
densi ty-vs-log  exposure  curve  which  takes  place  in 
photographic  film  and  the  limiting  of  signals  in  electronic 
devices.  The  film  densi ty-vs-log  exposure  of  figure  (2-8) 
is  a typical  example  of  the  limiting  functions  present  in 
all  image  transmission  and  storage  systems. 

In  Chapter  3,  a general  digital  color  analyzing  system 
model  was  described.  The  operation  of  the  model  on  an 
inout  color  _c  was  described  by  eg.  (3-17),  repeated  below 

v = HGFc 


where  the  model  components  are  defined  and  related  on  the 
block  diagram  of  figure  (3-2).  The  special  case  of  a film 
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and  scanner  color  system  is  also  indicated  in  figure  (3-2). 
The  film-scanner  color  analyzing  system  will  be  examined  in 
the  remainder  of  this  section,  in  order  to  give  a concrete 
and  important  example  of  a real  system  containing 
nonlinear ities.  The  concepts  developed  are  applicable  to  a 
much  wider  class  of  color  imaging  systems.  In  the  section 
whi:h  follows  the  film-scanner  discussion,  some  methods  of 
correcting  invertible  nonlinearities  will  be  discusser. 

The  type  of  color  photographic  film  which  will  be 
examined  is  reversal  color  film  [2],  which  produces  a three 
layer  positive  transparency.  In  a reversal  color  film, 
three  light-sensitive  layers  absorb  incident  light  at  a 
point  in  the  film  plane.  The  spectral  irradiance  at  the 
object  point  being  imaged  is  c(X).  The  spectral 
sensitivity  curves  of  the  layers  of  a typical  reversal 
color  film  are  shown  in  figure  (5-1).  The  sensitivities 
Sr(  X)  , Sg(A),  Sb(X)  of  the  three  layers  are  defined  by 

si(x)  " (5-4) 

for  i = R,G,B  where  E (X)  the  enerqy  per  unit  area  at  the 
film  plane  at  wavelength  which  is  necessary  to  generate 
some  fixed  density  value  after  development.  The  integrated 
exposure  values  of  the  fjlm  layers  are  given  bv 
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V'J> 


c(x)  s0(x>  t (x)  dx 

K O 


( 5-5a) 


r 


XG<C)  ‘ dG  / c<1)  sr,(>)  dA  ( 5— 5b) 


XB<C)  - “b 


/- 


C(X)  S_(X)  t (X)  dX 

D O 


( 5-5c) 


where  rQ  is  the  spectral  transmissivity  of  the  camera 

optics  and  dR,d^.,d^  are  Proportionality  constants  which 

contain  the  physical  parameters  which  qovern  the 

relationship  between  the  spectral  power  at  a point  in  the 

scene  being  photographed  and  the  spectral  energy  at  the 

film  plane.  These  parameters  include  the  F number  of  the 

camera  lens  and  the  duration  of  the  exposure.  This  allows 

the  exposure  values  to  be  defined  in  terms  of  the  spectral 

irradiance  c(A)  at  points  in  the  scene  rather  than  in  terms 

of  the  spectral  radiant  energy  at  the  film  plane.  In 

practice,  the  values  of  d^d^dg  are  usually  adjusted  so 

that  the  exposure  values  X ,X  ,X  are  equal  when  the  input 

RGB 

color  is  a chosen  reference  white.  Converting  eqs.  (5-5) 
to  a matrix  equation,  (see  Chapter  2)  yields 


x=Sc 


(5-6) 


where  the  3xn  matrix  S contains  dR,dG,dBand  ^(A)  as  well 


[ 


r 


as  SR(X)  , SG(X)  , and  SB(X)  . The  vector  x is  then  the 
exposure  vector  corresponding  to  the  color  vector  c and  the 
sensitivity  matrix  S. 

In  the  development  of  the  film,  the  layer  exposures 
are  converted  to  dye  densities  c,  m,  or  y accordinq  to 
relationships  such  as  those  shown  in  figure  (5-2).  These 
density  vs.  log  exposure  curves  are  approximate  in  that 
the  density  of  a developed  layer  is  dependent  not  only  on 
its  own  layer  exposure  as  shown  by  fiqure  (5-2)  , but,  to  a 
much  lesser  extent,  on  the  exposures  of  the  other  two 
layers.  This  phenomenon  is  called  the  interimage  effect 
13]*  T^e  densities  c,  m,  y are  the  dye  concentrations  of 
the  three  dye  layers  of  the  developed  transparency.  The 
spectral  densities  Dc(X),  Dm(X),  Dy(X)  (sometimes  called 
spectral  analytical  densities)  of  unit  concentration  of 
each  layer  dye  of  a typical  reversal  film  are  shown  in 
figure  (2-7).  The  dye  layers  are  cyan,  magenta,  and  yellow 
in  color  and  are  intended  to  control  the  transmission  of 
red,  green,  and  blue  light,  respectively.  The  spectral 
analytical  densities  of  the  three  superimposed  layers  is 

cDc(X)  + mDm(X)  + yDy(X)  . The  spectral  transmittance  r(X) 
of  the  transparency  is 


tt  = 10 


-cD  (X)  - mD  (X)  - yD  (X) 
c m y 


(5-7) 


When  dye  concentrations  of  c=m=y=l  are  present,  the 
transparency  spectral  analytical  density  is  the  upper  curve 
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of  fiqure  (5-3)  and  the  resulting  transparency  appears  to 
be  a neutral  qray  under  i llumin jtion  by  a reference* white 
liqht . 


* 


Equation  (5-7)  can  be  converted  to  vector 
the  following  definitions: 

rVVi 

w 

T_  = 

“W* 

Tel 


LyJ 


Vv 

• 

D (X  ) 
m 1 

• 

yv" 

• 

r/i 

• 

D (A  ) 

D (X  ) 

D (X  ) 

c n 

m n 

y n J 

n 

Equation  (5-7)  can  then  be  written  as 


Ti  = l° 


,T 

- d_^  w 


or,  with  slightly  irregular  matrix  notation, 


T = 10 


-Dw 


In  order  to  digitize  the  transparency, 


orm  using 


(5-8) 


(5-9) 


(5-10) 
a scanner 


samples  its  transmissivity  at  each  point  of  some  two 
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li 


rx  u 

■/  H» 


(X)  10 


- [cD  (X)  + raD  (X)  + yl)  (X)] 


m 


dX 


(5-12c) 


as  a function  of  the  dye  weiqhts  c,  m,  and  y. 

It  is  desired,  then,  to  solve  eo.  (5-12)  for  c,  m, 

and  y after  observinq  V vcy  and  Vg,  where  Dc , and  Dy 

are  known.  The  solution  is  difficult  because  of 

nonlinearities  and  undesired  absorptions,  two  of  the 

possible  sources  of  color  error  which  were  described  in 

Chapter  2.  The  nonlinear  relationships  among  c,  m,  y and 

V , V , and  V are  apparent  from  eq.  (5-12).  The 

R G B 

undesired  absorptions  occur  because  the  D , D . and  D 

c m y 

functions  are  nonzero  over  the  whole  spectral  region,  not 

just  over  the  appropriate  third  of  the  region.  If  D (X) 

m 

and  Dy(X)  were  zero  value  over  the  red  spectral  reqion, 

where  H (X)  is  nonzero,  then  eq.  (5-12a)  could  be  solved 

R. 

for  c independently  of  the  other  two  equations.  In 
general  , each  dye  absorbs  to  some  extent  over  the  whole 
spectral  region,  and  the  three  eouations  must  be  solved 
simultaneously.  Equation  (5-12)  can  be  converted  to  vector 
form 


v = Hi 


(5-13) 


where 
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v = V 


w VV‘ 

W w 

.W  VV_ 

Summarizing,  the  complete  input  color-to- 
chain  in  vector  notation  can  be  stated  as 


scanner  output 


-v  ~ Ht  = HG(w)  = HGF(x)  = HGFSc 


(5-14) 


where  H and  S are  linear  operations  which  can  be  described 
by  matrices  and  G and  F are  nonlinear  vector  operators. 


5.2  Exposure  Estimation  Method; 


The-  equations  relating  the  layer  exoosures  of  a color 
Photographic  film  to  the  corresponding  output  signals  of  a 
film  scanner  can  be  solved  exactly,  in  principle.  If  the 
film  dye  characteristics  Dc,  Dm,  Dy  and  the  scanner 
spectral  responses  HR,  HG,  HB  are  known,  then  the  three 

film  layer  densities  c,  m,  y can  be  obtained  by  inverting 
eg.  (5-12).  These  are  three  simultaneous  nonlinear 
equations  in  three  unknowns  c,  m,  and  y.  If  the  three 
D lo,  E curves  of  the  film  are  known,  then  they,  in  turn, 
can  be  used  to  obtain  the  layer  exposures  XE,  xG,  *B  fro. 
c,  m,  and  y. 
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In  Dractice,  there  are  two  reasons  why  an  exact 
solution  cannot  be  obtained.  First,  the  film  and  scanner 
characteristics  described  above  are  not  known  exactly;  they 
are  derived  from  experiments  which  are  subject  to  some 
deqree  of  measurement  error.  Secondly,  inversion  of  a set 
of  nonlinear  eauations  such  as  eq.  (5-12)  qenerally 
requires  iterative  solution.  Solution  alqorithms  may 
converqe  rapidly  to  the  solution  of  the  eauations,  but 
there  will  qenerally  be  some  residual  error  remaininq  after 
any  finite  number  of  iterations.  About  the  first  source  of 
error,  little  can  be  don^  except  to  obtain  the  best 
information  available  on  tne  film  and  scanner 
characteristics.  The  second  type  of  error,  computational 
error  in  the  solution  alqorithm,  can  be  made  almost 
arbitrarily  small  at  the  cost  of  complexity  of  the 
algorithm.  However,  since  the  inversion  algorithm  must  be 
executed  at  each  point  of  an  image  array  which  may  contain 
tens  or  hundreds  of  thousands  of  points,  the  computer  time 
required  for  its  execution  is  of  great  importance. 
Choosinq  the  optimal  inversion  algorithm  requires  a 
tradeoff  between  signal  error  and  computation  time. 


In  the  folic  -;nq  material,  a typical  set  of  film  dye 
transmissivity  and  density  vs.  log  exposure  curves  and  a 
set  of  color  scanner  response  curves  are  substituted  into 
eo . (5-12)  , and  several  solution  methods  of  various  kinds 
are  employed  in  order  to  compare  their  relative  accuracies 
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and  degrees  of  complexity. 

Tho  solution  methods  which  have  been  investigated  can 
be  assigned  to  one  of  three  general  types: 

1)  An  "exact"  solution  - Equation  (5-12)  can  be  solved 
with  arbitrarily  small  error  by  employing  a Newton-Raphson 
iteration  algorithm  which  converges  to  a solution  of  any 
desired  accuracy  in  a finite  number  of  steps. 

2)  Solution  by  approximation  of  the  original  equations 
- Equation  (5-12)  can  be  simplified  by  making 
approximations  which  convert  the  three  nonlinear  integrd 
equations  to  three  algebraic  equations  which  may  be  linear 
or  quadratic,  and  which  may  be  solved  analytically. 

3)  Solution  by  table  look-up  or  interpolation  - The 
equations  may  be  solved  by  substituting  values  of  film 
density  or  exposure  over  some  range  of  possible  values  into 
the  film-scanner  equations,  and  tabulating  the  resulting 
scanner  vo;.  ages.  To  keep  the  number  of  entries  in  the 
three  dimensional  table  within  reasonable  bounds,  coarse 
quantization  of  the  tabulated  values  can  be  combined  with 
linear  interpolation  between  the  tabulated  values. 
Alternatively,  the  whole  tabic  can  be  curve  fitted  by 
simple  interpolating  functions  such  as  polynomials.  The 
exposure  values  are  then  obtained,  within  certain  error 
limits,  by  inserting  the  scanner  signal  values  into  the 
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interpolating  fun  .tions  and  calculating  the  result. 


The  Newton-Raphson  algorithm  used  to  qive  an 
arbitrarily  exact  solution  of  eq.  (5-12)  is  a standard 
iterative  algorithm  for  solving  nonlinear  eauations  (4). 
In  this  algorithm,  an  approximate  solution  for  c,  m,  y is 
substituted  into  eq . (5-12)  to  obtain  a corresponding  set 

of  approximate  scanner  signals  V . V^.  The  differences 

Xx  Lr  B 

between  the  observed  V , V , V and  the  approximate  signals 

RGB 

are  then  used  to  generate  correction  terms  Ac,  Am,  Ay  to 
the  initial  approximate  values  of  c,  m,  y.  Thus, 


Ac 

3vr 

3c 

3vr 

3m 

!!e 

3v 

VR 

-*R 

Am 

= 

3vg 

3c 

!!<• 

3m 

3y 

VG 

"*c 

Ay 

!!i 

3c 

!!b 

3m 

3y 

_VB 

(5-15) 


The  process  can  be  iterated  until  the  desired  solution 
accuracy  is  achieved.  The  initial  approximate  solution  can 
be  obtained  using  any  of  the  algorithms  to  be  described  in 
the  following  material  on  aoproximate  solutions  of  eo. 
(5-12)  . 

Single  pass  solutions  of  ea.  (5-12)  can  be  obtained 
in  several  ways  by  imposinq  conditions  on  the  properties  of 
the  scanner  or  photographic  film  which  render  eq.  (5-12) 
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directly  solvable.  One  such  approximate  solution  is 
obtained  by  assuming  that  the  spectral  response  curves  of 
the  scanner  are  narrowband  and  that  the  density-log 
exposure  curves  of  the  film  are  linear  over  all  density 
values  of  interest.  The  last  condition  implies 


c = Qc  + >c  108  h 


m = q + -y  log  X- 
m m 6 G 


( 5-16a) 


( 5— 16b) 


y = ay  + log  XB  (5-16C) 

Substituting— ttrese  approximations  into  eqs . (5-12)  relates 

the  exposures  to  the  scanner  signals  V , V , V 

R G B 


4-  (X) 

4'  (a) 

Vj/  (X) 

= kR 

XL 

K(X) 

VX>  *R 

Y m 

XG 

XT 

XB 

dX 

( 3-1 7a) 

* C(X) 

^ (X) 

* (x) 

■ cX 

K(X) 

VX>  *r 

X m 
XG 

X y 

dX 

( 5-17b) 

v fK« 

*(X> 

* (x) 

4/  (X) 

' -»/, 

K(X) 

VX>  *R 

x m 
XG 

X y 
XB 

dX 

( 5— 17c) 

where 


vx> 


* (X) 

m 


y d (x) 

c c 


y d (x) 

mm 


vn 


y*> = \ Dy(x) 


- [a  D (X)  + a D (X)  + a D (X)] 

k(x)  =10  c c m m y y 


The  narrowband  assumption  imolies  that  the  scanner 
characteristic  H^,  H^.,  H are  nonzero  over  spectral  regions 
which  are  so  narrow  that  the  other  functions  of  X are 
approximately  constant.  This  assumption  is  very  good  for 
laser  scanners  but  less  good  for  broadbanded  scanner 
characteristics.  Under  the  narrowband  assumption,  ea. 
( 5-17 ) becomes 


V (X  ) Y (X_)  v (X_) 


VV  „ W y W 

G XB 


VG  - AC  **  ' ' X"  X 


iJK)  ».«.) 

B "B  "R  "G  "B 


V.  - A.  X„  X B 


where 


R ’ kR  K(V  [ V VX> 
al 


dX 


AG  “ kG  K(V  / "G 
al 


(A)  dX 


( 5-18a) 


( 5-18b) 


(5-18c) 
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AB  = kB  K(V 


, (A)  dA 


and  VXG  ,lB  are  the  cenfcroid  wavelengths  of  the  scanner 
functions  hr,  Hq,  Hb,  respectively.  Taking  the  logarithm 
of  each  side  of  ea.  ( 5-18)  gives  the  matrix  eauation 


VV 

VV 

Vc(V 


m R 

W 

in  G 

VV 

m B 


VV 

W 

W 


X + £n  A 

G 


(5-19) 


Inverting  *"his  equation  and  exponentiating  gi 


gives  the 


* A A 


estimated  exposures  XD,  x^,  x„. 

K Lr  B 


Another  approx imation  can  be  made  in  eos.  (5-12) 
which  renders  the  equations  solvable  in  closed  form.  The 
necessary  approximation  is  that  the  film  dyes  are  ideal 
b;  >ok  dyes;  that  is,  the  three  layer  absorption  curves  are 
spectrally  nonoverlapping  and  are  constant  with  wavelength 
over  their  absorption  regions  as  shown  in  figure  (2-8). 

under  this  assumption,  eg.  (5-12)  reduce  to  the  following 
equations 


VR  = bll  10  C + b12  10 


™ -yn 

+b1310  y 


( 5-20a) 


4 


-cD  -mD  -yD 

Vg  = b31  10  c + b32  10  m + b33  10  y (5-20C) 

where  D , D , D are  the  constant  ideal  dye  absorptions 
c m y 

over  their  absorption  regions  and  where 

b = f Uj  H±(X)  dX 
‘,XLj 

for  i=R, G ,B ; j=c ,m , y and  where  X,  and  X define  the  lower 

lc  uc 

and  upper  spectral  limits  of  the  cyan  dye  absorption 
region,  with  similar  definition  for  the  magenta  and  yellow 
absorption  region.  In  matrix  form,  ea.  (5-20)  becomes 


Inverting  this  matrix  eauation,  taking  the  base  10 

logarithm  of  the  components  of  the  resulting  vector,  and 

dividing  by  the  appropriate  constants  -D  , -D  , or  -D 

r c m y 

gives  the  approximate  values  of  c,  m,  and  y under  the  ideal 
dye  approximation.  The  accuracy  of  the  answer  is 


1 


determined  by  the  closeness  of  the  actual  dye  absorption 
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curves  to  the  nearest  set  of 

ideal 

dye 

curves.  As 

was 

stated  in  Chapter  2, 

real 

dyes 

have 

some  degree 

of 

absorption  in  undesired 

spectral  req 

ions , 

and  do  not 

have 

constant  absorption  over 

the 

desired 

absor 

ption  region. 

Inversion 

of 

the  system 

of  eauations 

defined 

by 

the 

density-log 

exposure  curves  and  eg. 

(5-12) 

can 

be 

accomplished 

in 

other 

approximate 

ways  by 

f 

irst 

substituting 

a 

range  of 

exposure  values  x in 

to 

the 

equations  and  tabulating  the  resulting  scanner  signals  y. 

Determination  of  x from  y can  then  be  accomplished  by  any 
of  the  following: 

1)  choosing  the  tabulated  v nearest  to  the  observed  v 
and  lookino  up  the  corresponding  x in  the  table. 

2)  curve  fitting  the  x values  to  the  y values  using  a 
set  of  algebraic  interpolating  functions,,  thus 
obtaining  estimated  values  of  x from  observed  values 
°f  v by  inserting  the  observed  v into  the 
interpolating  function  and  calculating  the 
corresponding  value  of  x. 

3)  using  a combination  of  the  above. 

For  example,  a coarsely  quantized  table  of  x vs.  v 
might  be  used  along  with  a simple,  perhaps  linear, 
interpolation  algorithm  for  obtaining  values  when  y is  not 

equal  to  one  of  the  relatively  small  number  of  y's  in  the 
coarse  lookup  table. 
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The  curv’  fitting  and  interpolation  algorithms  can  be 
simplified  by  using  the  logarithms  of  the  variables  rather 
than  the  variables  themselves,  since  the  logarithms  of  the 
components  of  ^ and  are  very  nearly  linearly  related. 
This  may  be  inferred  by  observing  that  under  the  narrowband 
scanner  assumption,  eas.  (5-12)  simplify  to  egs.  (5-19) 
which  are  exactly  linear  in  the  logarithms  of  x and  v. 

5.3  Exposure  Estimation  Results 

In  this  section,  the  exposure  estimation  algorithms 
which  have  been  described  are  compared  by  testing  them  with 
exposure  and  scanner  signal  values  obtained  using  a 
computer  simulation  of  a typical  film-scanner  color 
analyzing  system.  The  film  properties  are  those  of  a 
tyDical  reversal  color  film  (Kodak  Ektachrome-X) . The 
density  vs.  log  exposure  and  layer  dye  spectral  densities 
of  the  film  are  shown  in  figures  (5-2)  and  (2-7).  The 
scanner  simulation  represented  a flying  spot  scanning 
system  consisting  of  a cathode  ray  tube  light  source,  a 
photomultiplier  tube  sensor  with  an  S20  sensitivity 
characteristic,  and  color  filters  which  are  Kodak  Wratten 
filters  number  25  (red),  58  (green)  and  47B  (blue).  In 
addition,  the  spectral  absorption  of  a typical  set  of 
lenses  was  incorporated.  The  overall  spectral  responses  of 
the  simulated  scanner  in  its  red,  green,  and  blue  channels 
are  shown  in  figure  (5-3). 
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The  algorithms  which  were  tested  as  exposure 
estimators  were  the  following: 

1)  Newton-Raphson 

2)  Narrowband 

3)  Ideal  Dye 

4)  Quadratic  Curve  Fit 

5)  Table  Look-up  with  Iterative  Correction 

The  parameters  of  the  algorithms  were  adjusted  to  the 
particular  film-scanner  system  being  considered  in  the 
following  ways: 


1)  Newton-Raphson  - The  Newton-Raphson  solution  method 
was  carried  out  for  each  input  color  until  the  change  in 
estimated  exposure  values  for  iteration  was  sufficiently 
small.  The  iterations  were  stopped  when  the  RMS  fractional 
change  in  the  estimated  exposures  averaged  over  the  three 
exposure  values,  was  less  than  0.01.  This  was  achieved,  on 
the  average,  after  about  six  iterations. 

2)  Narrowband  - The  narrov;band  spectral  sensitivities 
were  placed  at  the  centroids  of  the  three  scanner  response 
curves,  which  are  at  wavelengths  of  610,  535,  and  440 
nanometers  for  the  red,  green  and  blue  scanner  channels, 
respectively. 

3)  Ideal  Dye  - The  dye  spectral  density  functions 
which  were  assumed  under  the  ideal  dye  model  were  of  the 
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ideal  rectangular  shape  as  shown  in  fiqure  (2-1)  where  the 
wavelengths  seoarating  the  three  dye  absorption  curves  were 
chosen  to  be  the  intersection  wavelengths  of  the  actual 
dyes,  and  the  constant  densities  of  the  ideal  dyes  were 
selected  so  that  their  densities  integrated  over  all 
wavelengths  equalled  the  integrated  densities  of  the  actual 


dyes . 


4,  Ouadratic  Curve  Fit  - A ouadratic  polynomial 
exposure  estimator  was  derived  by  obtaining  a least  squares 
curve  fit  between  a set  of  exposure  vectors  x and  the 
corresponding  scanner  signal  vectors  v,  obtained  by 
substituting  the  x vectors  into  the  equations  which  model 
the  film  and  scanner  Droperties 

v = HGF(x) 

The  oolynomial  expression  which  approximates  the  components 
of  x from  the  components  of  v was  selected  to  be  a 
quadratic  expression  in  the  logarithms  of  the  components 

(loB  x±)  - l ay  log  Vj  + ^ b1Jk  log  V(  log  V|(  + c. 

where  i,j,k  - R,G  or  B.  This  was  done  since  it  was  known 
that  the  relationship  between  the  log  v and  log  x 
quantities  is  exactly  linear  in  the  narrowband  scanner  case 
(equation  5-14),  and  it  was  hoped  that  many  of  the 
quadratic  coefficients  could  be  neglected  in  the  general 
scanner  approximation.  Inis  indeed  turned  out  to  be  the 
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case,  and  it  was  found  that  the  quadratic  coefficients 

which  were  the  largest  were  those  corresponding  to  the 

channel  being  approximated.  That  is,  in  approximating  log 

2 

Xr,  coefficient  of  the  (log  VR)  term  was  much  larger 

than  that  of  the  (log  VRlog  VG)  term  or  the  (log  V^)2  term. 
This  allows  the  approximation  to  take  the  simple  form 


(log  XA)  = | a±.  log  Vj  + bi  (log  V1)2  + Ci 


Expressed  in  matrix  form,  this  becomes 


The  a,  b,  and  c coefficients  were  determined  by  least 
squares  curve  fittinq  over  125  points,  5 in  each  dimension 
of  x.  The  five  values  each  of  log  X_,  log  X-  and  log  X 
were  chosen  to  cover  in  equal  increments  most  of  the  linear 
region  of  the  density-log  exposure  curves  of  the  film. 

5)  Linear  Correction  Schemp  using  Table  Look-up  for 
Initialization  - An  efficient  iterative  method  for  solving 
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the  layer  dye  weiqhts  c,  m,  v was  derived  from  a method 
Of  Eugene  Allen  |5)  for  solving  a related  oroblem,  that  of 
determining  the  dye  weights  which  give  a colorimetric  match 
to  a given  sample.  Details  of  the  method  can  be  found  in 
reference  15).  The  only  change  required  for  the 
film-scanner  problem  is  the  substitution  of  the  scanner 
Characteristics  for  the  color  matching  curves  used  by 
Allen.  The  method  essentially  linearises  the  relations 
between  c,m,y  and  the  scanner  siqnals  about  the  initial 

values  of  c,m,y  and  generates  a linear  corrective  term 
which  is  added  to  the  initial  values. 

The  method  was  tested  using  as  initial  estimates 
obtained  from  a coarse  three-dimensional  (5x5x5) 
loox-up  table, followed  by  one  step  of  linear  correction 
using  the  Allen  method.  The  table  was  obtained  by 

inputting  a range  of  125  density  vectors  „,  spaced 
uniformly,  into  the  film-scanner  simulation  and  forming  a 
three-dimensional  table  of  the  output  vectors  v and  the 
input  w's.  Given  an  observed  v , the  nearest  smaller  y 
components  in  the  table  were  selected,  and  the" 
correspondinq  w was  used  as  an  initial  point. 

The  results  of  a computer  simulation  testing  of  the 
various  exoosure  estimation  algorithm  are  summarized  in 


Table  (5-1),  along  with  th 


e results  of  using  no  correction 


film  and  scanner  nonlinearities.  The  simulati 


on 
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Table  (5-1) 

Exposure  Estimator  Error  Performance 


Exposure 
Estimation 
Alqor ithm 

Narrowband 

Approximation 


Ideal  Dye 
Approximation 


Quadratic 
Curve  Fit 


Table  Lookup 
With  Linear 
Correction 


Newton-Raphson 
Iterative  Method 


No  Correction 
(Scanner  Signals 
Treated  as 
Exposures) 


Normalized 

Exposure 

Error 


0.0392 


0.0662 


0.0110 


0.0213 


0.0061 


0.1045 
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modelled  the  whole  photoqraohic  film  and 
system  whose  character isti 


color  scanner 


cs  are  given  by  the  curves  of 
ficures  (2-7,, ,5-1,,  ,5-2,  and  ,5-3,.  The  input  colors 

were  the  ten  reflectivities  of  figure  (4-1)  illuminated  bv 
Cl E Illuminant  C. 

The  results  indicate,  as  expected,  that  the  more 
complex  methods  were  more  accurate.  Of  the  two  simplest 
methods,  the  narrowband  approximation  was  more  accurate 
than  the  ideal  dye  approximation,  althouqh  both  were  much 
better  than  no  correction  at  all.  The  sliqhtly  more 
complex  quadratic  curve  fit  was  better  than  either  of  the 
simpler  methods,  bv  a factor  of  four  or  more,  and  was  also 
more  accurate  than  the  more  complex  table  lookup  with 
linear  correction.  The  table  lookup  with  one  step  of 
correction  was  far  surpassed  by  the  Newton-Raohson  method, 
which  executed  six  iterative  steps  per  solution. 
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6.  Related  Applications  of  Digital  Color  Methods 

The  vector  space  description  of  color  analysis  which 
was  described  in  Chapter  3 and  the  generalized  inverse 
methods  which  were  used  for  tristimulus  estimation  in 
Chapter  4 can  be  applied  to  a wide  variety  of  color 
problems.  in  this  chapter,  two  problems  will  be  discussed. 
The  first,  spectral  calibration  of  an  optical  sensoi  using 
readings  from  test  samples,  is  a practical  problem  which 
arises  when  direct  spectrophotometric  measurement  of  the 
sensor  spectral  characteristic  is  impossible.  The  second 
problem,  that  of  computer  generation  of  spectral  waveforms 
with  desired  properties,  arises  in  the  simulation  of  color 
or  multi-spectral  sensing  and  recording  systems. 

6.1  Estimation  of  Color  Scanner  Spectral  Characteristics 


The  spectral  response  of  an  optical  sensing  system, 
such  as  a color  image  scanner  or  a densitometer,  is  often 
difficult  to  measure  directly.  The  spectral  response  is 
typically  the  product  of  a series  of  spectral 
characteristics  of  the  system's  component  parts,  such  as 
the  emission  characteristic  of  the  light  source,  the 
transmissivity  of  the  system  optics  and  the  response 
characteristic  of  the  sensor.  The  spectral  response  of  the 
system  as  a whole  can  be  measured  directly  only  by  placing 
a wavelength  variable  narrowband  device,  such  as  a 
calibrated  monochromator,  in  the  system  light  path.  This 
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may  be  impossible  because  of  physical  size  limitations, 

accuracy  requirements,  or  other  reasons.  An  alternative 

approach  is  to  take  readings  with  the  scanner  using  a 

series  of  test  color  samples  whose  spectral  characteristics 

are  known,  and  to  reconstruct  the  scanner  spectral  response 

from  these  readings.  The  vector  equation  relating  the 

scanner  characteristic  h,  the  i th  color  sample 

transmissivity  c.  and  the  corresponding  scanner  output  x 
— 1 i 

is  given  by 


x 


i 


h + n 


(6-1) 


where  c and  h are  Qxl  vectors  of  quadrature  samples  of  the 
test  color  sample  transmissivity  curve  and  the  scanner 
response  curve,  respectively.  The  observation  noise  error 
on  the  i th  observation  is  given  by  ni . By  cor  bining  the  P 

observations  x.  (i=l, ,P)  into  a Pxl  vector  x,  eq.  (6-1) 

becomes  the  matrix  equation 


x = C h + n (6-2) 

T . 

where  c^  is  the  i th  row  of  the  PxQ  matrix  C.  Since  the 
number  of  quadrature  mesh  points  0 is  usually  much  larger 
than  the  number  of  test  samples  P,  eq.  (6-2)  is 
underdetermined,  and  the  tools  which  were  used  to  solve  the 
same  equation  (eq.4-1)  in  Chapter  4 can  be  again  applied. 
The  mathematical  solution  for  h in  eq.  (6-2)  is  equivalent 
to  solving  for  c in  eq.  (4-1)  because  of  the  symmetry 

127 


I 


between  c and  h in  eq.  (6-1).  In  the  tristimulus  problem 
the  sensor  response  was  known  and  the  input  color  was  being 
estimated.  Now,  the  input  color  is  known  and  the  sensor 
characteristic  is  being  estimated. 

The  three  types  of  solutions  developed  in  Chapter  4 
are  the  pseudo-inverse  solution,  the  generalized  inverse 
with  a smoothness  constraint,  and  the  Wiener  solution.  All 
three  are  applicable  to  the  scanner  spectral  characteristic 
problem.  The  smoothness  constraints  imposed  on  the 
estimated  characteristic  by  the  constrained  generalized 
inverse  solution  and  the  Wiener  solution  are  desirable 
since  the  spectral  response  of  a broad  banded  color  sensor 
can  be  assumed  a priori  to  be  a smooth  function.  The  three 
solutions  to  eq.  (6-2)  are 

(1)  pseudo-inverse  solution 

- T T -1 

h = C (C  C ) x (6-3) 

(2)  generalized  inverse  solution  with  smoothness 
constraint 

hs^-1  CT  (.C  N_1  CT)_I  x (6-4) 

where  N is  a suitably  chosen  smoothing  matrix  such  as 
that  of  equation  (4-29),  which  minimizes  the  average 
squared  difference  over  all  solutions  to  x=Ch. 

(3)  Wiener  solution 
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i = ”^+^T  (CmJ  + K)'1  (x-Ci^)  (6-5) 

where  m,  is  the  mean  vector  and  M is  the  covariance 
matrix  of  the  vector  random  process  from  which  h is 
assumed  to  be  a sample,  and  K is  the  covariance  of 
the  observation  noise. 

The  three  solutions  were  used  in  the  spectral 
calibration  of  an  Optronics  Model  S2000  flat  bed  scanning 
microdensitometer.  The  test  samples  were  two  sets  of 
spectrally  selective  filters,  each  set  covering  the 
spectral  region  of  interest,  roughly  400  to  700  nanometers 
in  wavelength.  The  first  set  consisted  of  nine  absorption 
filters,  foui  gelatin  (Kodak  Wratten)  filters  and  five 
glass  absorption  filters.  Their  transmissivities  are  shown 


in  figure 

(6-1)  . 

These  were 

obtained 

by  sampling 

the 

continuous 

transmissivities 

at 

80 

points 

at  5 

nm. 

intervals . 

The 

resulting  C matrix 

was  therefore 

9x80  . 

The 

second  set  of  filters  consisted  of  16  Oriel  narrowband 
interference  filters,  shown  in  figure  (6-2).  These  are 
spectrally  spaced  from  400  to  700  nanometers  in  increments 
of  20  nanometers  and  have  half  power  bandwiaths  of  10 
nanometers.  The  transmissivities  were  sampled  at  160 
points  in  steps  of  2.5  nm . (380  to  777.5  nm.),  giving  a 

16x160  C matrix . 

The  N matrix  used  in  the  constrained  generalized 
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inverse  solution  was  the  smoothing  matrix  of  equation 
(4-29)  and  the  Wiener  solution  assumed  a zero  mean  vector, 
a first  order  Markov  covariance  matrix,  and  negligible 
observation  noise. 

Figure  (6-3)  shows  the  estimated  response  for  the 
three  types  of  estimator  using  the  measurements  taken  with 
the  absorption  filters.  Figure  (6-3a)  shows  the 
pseudo-inverse  estimate,  figure  ( 6—  3b)  shows  the 
constrained  generalized  inverse  estimate  and  figure  (6-3c) 
shows  the  Wiener  solution  with  three  different  values  of 
assumed  interelement  correlation.  Figure  (6-4)  gives  the 
estimates  obtained  using  the  same  types  of  estimators  but 
with  data  obtained  using  the  narrowband  interference 
filters.  The  Wiener  estimates  with  the  highest  correlation 
values  (0.9  or  0.95)  give  the  smoothest  curves  obtained 
with  both  the  absorption  and  interference  filter  data,  and 
may  be  presumed  to  be  the  best  estimates  of  the  true 
response.  The  interference  filter  estimate  might  be 
expected  to  be  more  accurate  than  the  absorption  filter 
estimates,  since  there  were  more  of  the  interference 
filters  and  since  their  passbands  do  not  overlap 
significantly,  making  eq.  (6-2)  extremely  well 
conditioned.  The  "true"  spectral  response  curve  is  not 
available,  however,  but  since  the  Wiener  estimates  with  the 
two  filter  sets  are  in  fairly  good  agreement,  it  may  be 
concluded  that  both  estimated  responses  are  a good 
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Estimated  scanner  response,  Wiener  inverse,  absorption  filters 


RELATIVE  RESPONSE 


SUU  500  600 ^ 700 L 

WAVELENGTH,  n.m. 

Figure  (6-4a)  Estimated  scanner  response,  pseudo-inverse 
i'  fc^-ference  filters 


500  600 70Q ‘ 

WAVELENGTH,  n.m. 

Figure  (6-4b)  Estimated  scanner  response,  smoothing  inverse 

interference  filters  ’ 


approximation  to  the  actual  spectral  response 


6.2  Computer  Generation  of  Spectral  Waveforms 

The  computer  simulation  of  color  and  multispectral 
sensing  and  recording  systems  often  requires  the  generation 
of  spectral  waveforms  which  simulate  the  spectral  power 
distributions  existing  at  points  in  the  sensor  field  of 
view.  it  may  be  desired  that  the  n-vector  which  represents 
the  sampled  spectral  waveform  possess  certain  properties 
expressible  as  linear  equality  constraints.  For  example, 
it  might  be  desired  to  generate  a spectral  waveform,  or 
color,  whose  tristimulus  values  are  specified.  m 
addition,  the  generated  waveform  should  exhibit  properties 
which  are  characteristic  of  spectral  waveforms  existing  in 
nature,  such  as  smoothness  and  non-negativity.  These 

requirements  are  nearly  identical  to  those  of  the  waveforms 
generated  in  the  tristimulus  estimation  methods  of  Chapter 
4.  The  difference  is  that  the  matrix  S in  the  constraining 
equation  x=Sc  need  not  represent  a set  of  real  sensor 
characteristics,  but  instead  may  represent  any  linear 

operation  on  the  waveform  c,  such  as  a tristimulus 
constraint  of  the  form  t=Tc . 

The  generation  of  spectral  waveforms  metameric  to  a 
given  color  is  a special  case  of  waveform  generation  which 
has  been  the  subject  of  some  study.  This  is  equivalent  to 
generating  waveforms  with  desired  tristimulus  values.  The 
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methods  which  have  been  derived  usually  construct  the 
waveform  as  weighted  sums  of  three  or  four  simple  functions 


such  as  narrowband  functions  (see  reference  [1], 
p.  344-357)  for  a summary  of  methods  to  1967).  A recent 
method  developed  by  Takahama  and  Nayatani  [2]  is  derived 
using  calculus  of  variations,  and  constructs  the  function  c 
as  a weighted  sum  of  color  matching  curves.  The  Takahama 
algorithm  expressed  in  sampled  data  form  is  equivalent  to 

the  pseudo-inverse  solution  of  the  constraining  equation, 

+ 

c=T  t.  To  generate  many  metameric  waveforms  satisfying  the 
constraint,  c is  varied  and  the  solution  c=c  +T+(t-Tc  ) is 
the  solution  of  t=Tc  which  is  nearest  in  the  least  squares 
sense  to  c Q . 

The  Wiener  and  smoothing  inverses  described  in  Chapter 
4 can  be  applied  to  the  generation  of  metameric  waveforms 
with  some  advantages.  The  imposition  of  smoothing  can,  for 
example,  generate  waveforms  which  are  very  broad  and  smooth 
while  still,  satisfying  t=Tc . The  general  waveform 
expression  is  given  by 

c = c + MTT(TMTT)-1  (t  - T c ) (6-6) 

— — o — — — — o 

where  t is  the  desired  tristimulus  vector,  T is  the  matrix 

whose  rows  are  color  matching  curves,  cq  is  an  arbitrary 

waveform,  and  M is  a smoothing  matrix.  Waveforms  generated 

by  this  algorithm  are  shown  in  Figure  (6-5).  A test  color 

is  shown  as  a dotted  line,  with  waveforms  metameric  to  the 
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test  color  (with  respect  <.0  illuminant  C)  shown  as  solid 

I lines.  The  metameric  waveforms  were  generated  using 

eq.(6-6)  with  £^=0  and  the  Markov  matrix  of  eq.(4-64) 
replacing  M.  The  values  of  p were  0.0,  0.8,  and  0.95. 
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7.  Conclusions  and  Topics  for  Further  Study 


This  dissertation  has  presented  a variety  of 
techniques  for  digitally  restoring  digital  color  images. 
The  methods  which  were  treated  included  ways  of  correcting 
for  color imetrically  imperfect  sensors  and  correcting 
errors  introduced  by  color  system  nonlinearities  and 
undesired  film  absorption.  In  addition,  methods  for 
estimating  the  spectral  response  of  a broadband  sensor  from 
sample  readings  and  for  computer  generation  of  waveforms 
with  desired  colormetric  properties  were  treated. 


The  correction  for  sensor  spectral  imperfections,  or 
tristimulus  estimation,  was  best  accomplished  using  a 
wiener  estimation  algorithm  which  imposed  a large  smoothing 
effect  on  the  spectral  waveforms  generated  as  intermediate 
steps  in  the  estimation  of  tristimulus  values.  The  RMS 
error  in  UCS  chromaticity  space  was  0.011  averaged  over  ten 
test  colors  using  color  film  taking  sensitivities.  The 
method  has  the  advantages  of  simplicity,  linearity, 
optimality  (under  a mean  squared  tristimulus  error 
criterion)  and  generality.  The  property  of  generality 
assures  that  the  estimator  can  easily  be  re-optimized  when 
the  taking  sensitivities  or  the  assumed  statistical 
properties  of  the  input  colors  are  changed.  Although  the 
method  was  restricted  in  this  report  to  the  estimation  of 
three  tristimulus  values  from  three  outputs  of  imperfect 
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spectral  sensors,  it  is  easily  generalized  to  other 
dimensionalities.  The  outputs  of  m sensors  of  arbitrary 
spectral  response  could  be  used  to  estimate  the  outputs  of 
n other  arbitrary  sensors,  where  m is  greater  than,  equal 
to,  or  less  than  n.  Applications  of  the  method  might 
include  the  filling  of  spectral  gaps  in  multi-spectral 
sensor  data,  or  the  generation  of  "true  color”  imagery  from 
multi-spectral  data.  All  that  is  reauired  is  that  the 
spectral  correlation  properties  of  the  incoming  radiation 
are  known  with  some  degree  of  accuracy,  and  that  there  is 
significant  correlation  between  the  observed  spectral 
regions  and  those  being  estimated. 

The  material  dealing  wich  correction  of  color  system 
nonlinearities  and  undesired  absorption  showed  that  the 
resulting  color  errors  can  be  greatly  reduced,  for  a 
typical  film-scanner  system,  by  using  relatively  simple 
computer  algorithms.  Simple  linear  and  quadratic 
approximations  to  an  exact  solution  proved  capable  of 
reducing  errors  in  estimated  film  exposures  by  a factor  of 
about  ten,  for  the  system  which  was  simulated. 

The  chapter  on  applications  related  to  digital  color 
restoration  used  as  its  primary  mathematical  tool  the 
generalized  matrix  inverse,  used  previously  in  the  chapter 
on  tristimulus  estimation.  In  fact,  the  most  important 
single  result  of  this  report  may  be  the  demonstration  of 
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the  great  utility  of  the  generalized  inverse  as  applied  to 
color  systems.  The  applications  treated  in  this 
dissertation  included  tristimulus  estmation,  estimation  of 
sensor  spectral  responses,  and  computer  generation  of 
spectral  waveforms.  The  generalized  matrix  inverse  is 
applicable  to  a great  many  other  problems  connected  with 
color  and  multi-spectral  analyzing  and  imaging  systems. 
Any  problem  requiring  the  solution  or  inversion  of  a set  r.f 
linear  vector  equations,  subject  to  the  minimization  of  a 
vector  quadratic  form,  is  a candidate  for  generalized 
inverse  methods.  This  is  true  whether  the  equations  are 
deterministic  or  stochastic,  overdetermined , 
underdetermined  or  exactly  determined.  Typical  problems 
might  include  the  estimation  of  the  spectral  properties  of 
sample  reflectivities,  sensors,  illuminants,  atmospheric 

transmissivity,  lens  oi  filter  absorption  characteristics, 
etc . 


A fruitful  area  for  further  study  is  in  determining 
the  best  quadratic  forms  for  use  as  error  criteria  with 
linear  estimation  techniques.  A quadratic  form  which  best 
measures  the  departure  of  a spectral  waveform  from  the 
smoothness  or  correlation  properties  of  natural  object 
colors  would  be  useful.  The  optimal  quadratic  forms  for 
converting  the  vector  of  tristimulus  errors  or  the  vector 
of  spectral  distribution  errors  to  a number  measuring  the 
total  perceived  color  error  would  also  be  of  some  benefit 
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in  color  estimation  problems. 


In  conclusion,  it  is  hoped  that  the  contents  of  this 
dissertation  are  of  some  value  in  advancing  the  science  and 
art  of  digital  color  image  processing.  The  field  of 
digital  color  image  processing  is  in  its  infancy,  and  will 
doubtless  evolve  far  beyond  its  current  state.  The  uneasy 
marriage  of  colorimetry  and  linear  system  theory, 
characteristic  of  the  methods  described  here,  will  someday 
be  supplanted  as  newer,  more  complex  and  more  accurate 
mathematical  models  of  the  human  visual  systems  are 
created.  Still,  in  spite  of  the  obvious  limitations  of 
linear  estimation,  quadratic  error  criteria,  and  present 
day  colorimetry,  methods  such  as  those  described  here  are 
effective.  With  the  increasing  power  of  digital 
computation  the  digital  processing  of  color  imagery  will 
become  even  more  effective  and  more  widespread. 
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